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EXECUTIVE SUMMARY 


In the rapidly changing world of the 21st century, science literacy is an essential goal for all of 
our nation’s youth. Through science education, children come to understand the world in which 
they live and learn to apply scientific principles in many facets of their lives. In addition, our country 
has an obligation to provide young people who choose to pursue careers in science and technology 
with a strong foundation for their postsecondary study and work experience. The nation’s future 
depends on scientifically literate citizens who can participate as informed members of society and as 
a highly skilled scientific workforce, well prepared to address challenging issues at the local, national, 
and global levels. Recent studies, including national and international assessments, indicate that our 


schools still do not adequately educate all students in science. 


Science seeks to increase our understanding of the natural world through empirical evidence. 
Such evidence gathered through observation and measurement allows for an explanation and 
prediction of natural phenomena. Hence, a scientifically literate person is familiar with the natu- 
ral world and understands key facts, concepts, principles, laws, and theories of science, such as the 
motion of objects, the function of cells in living organisms, and the properties of Earth materials. 
Further, a scientifically literate person can connect ideas across disciplines; for example, the con- 
servation of energy in physical, life, Earth, and space systems. Scientific literacy also encompasses 
understanding the use of scientific principles and ways of thinking to advance our knowledge of the 
natural world as well as the use of science to solve problems in real-world contexts, which this docu- 


ment refers to as “Using Technological Design.” 


The National Assessment of Educational Progress (NAEP) and its reports are a key measure in 
informing the nation on how well the goal of scientific literacy for all students is being met. The Sci- 
ence Framework for the 2019 National Assessment of Educational Progress sets forth the design of the 
NAEP Science Assessment. The 2019 NAEP Science Assessment will use the same framework used 
in 2009. The 2009 NAEP Science Assessment started a new NAEP science trend (i.e., measure of 
student progress in science), and the 2019 NAEP Science Report Card will include student perfor- 
mance trends from 2009 to 2019. Trends in student science achievement were reported from 1996 
to 2005 as well. However, the trend from 1996 to 2005 was not continued due to major differences 
between the 2005 and 2009 frameworks. The 2009 - 2019 framework represents a unique opportu- 


nity to build on key developments in science standards, assessments, and research. This document is 


ix | 


intended to inform the general public, educators, policymakers, and others about what students are 


expected to know and be able to do in science as part of The Nation’s Report Card, a program of the 
US. Department of Education (ED) that reports on NAEP findings. 


In 1988, Congress created the National Assessment Governing Board to set policy for NAEP. 
The National Center for Education Statistics (NCES, a division of ED) carries out NAEP. As the 
ongoing national indicator of the academic achievement of U.S. students, NAEP regularly collects 
information on representative samples of students in grades 4, 8, and 12 and periodically reports 
on student achievement in reading, mathematics, writing, science, and other subject areas. NAEP 
scores are always reported at the aggregate level, not for individual students or schools. (By law, 
NAEP cannot report results for individual students.) For science, NAEP results are reported at the 
national and state levels and for a number of large urban districts. The district reports are provided 


for urban school systems that volunteer for the Trial Urban District Assessment component of 
NAEP. 


NAEP produces comparative student achievement results according to demographic factors 
such as gender, race/ethnicity, and geographic region. Results are also provided in terms of student, 
teacher, and school background variables related to science achievement. Taken together, this infor- 
mation from NAEP helps the general public, educators, and policymakers make informed deci- 
sions about education. Interested individuals can access performance results and released questions 


through NAEP reports and websites. 


Beginning in 2009 a new framework to guide the science assessment was necessary for several 
reasons: publication of National Standards for science literacy, advances in both science and cogni- 
tive research, growth in national and international science assessments, and increases in innovative 
assessment approaches. This framework presents the content to be assessed as well as the conceptual 
base for the assessment. It is intended for a general audience. A more detailed technical document, 
Science Assessment and Item Specifications for NAEP, is a companion piece. The specifications doc- 
ument provides detailed information on the content to be assessed, item development, and other 
aspects of assessment development and administration. The audience for the science specifications is 


NCES and the NAEP assessment development contractors. 


Key Features 


This framework is the result of extraordinary effort and commitment by hundreds of individuals 
across the country, including some of the nation’s leading scientists, science educators, policymakers, 
and assessment experts. Under contract to the Governing Board, WestEd and the Council of Chief 
State School Officers (CCSSO) spent 18 months developing the framework; this process involved 


committees, regional hearings, and other public forums. The Governing Board also engaged an 


external review panel to evaluate the draft framework and convened a public hearing to gather add 


tional input during the development process. 


The 2009 - 2019 framework incorporates the following key features: 


e Its design is based on widely accepted national science standards and assessments in addi- 
tion to state curriculum standards. However, it is intended to inform development of an 
assessment, not to advocate for a particular approach to instruction or to represent the 
entire range of science content and skills. 

e In view of the need to keep the United States and its youth internationally competitive in 
science and technology, the framework development process gave special consideration 
to international assessment frameworks, such as those for Trends in International Mathe- 
matics and Science Study (TIMSS) and Programme for International Student Assessmen 
(PISA). 

e The breadth of the science principles represented in the source materials made it necessar 
to focus on the foundational and pervasive knowledge within each discipline and to pare 


down the science content to be assessed. 
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e The framework is based on scientific knowledge and processes derived from tested explana- 


tions and supported by accumulated empirical evidence. Explanations of natural phenom 


ena that rely on nonscientific views are not reflected in the framework. 


e Science content is presented in detailed, grade-specific charts that also allow the reader to 


see the progression in complexity of ideas across grades. 


e Every attempt has been made to be free of error in describing the science content. The lan- 


guage used strives to be accurate but not technical so as to make the framework accessible 
to a wide audience. 

e The focus is on students’ conceptual understanding, that is, their knowledge and use 
of science facts, concepts, principles, laws, and theories. Students’ abilities to engage in 
some components of scientific inquiry and technological design are also reflected in the 
framework. 


e New types of items are recommended, including the use of interactive computer tasks. 


Content of the Framework 


The framework describes the science content and science practices that form the basis for the 
NAEP Science Assessment. It also discusses item distribution and types of items, as well as draft 
achievement levels. Finally, it recommends several small-scale, special studies to be considered in 


conjunction with the science assessments. 
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Science Content 


The science content for NAEP is defined by a series of statements that describe key facts, con- 


cepts, principles, laws, and theories in three broad areas: 


e Physical Science 
e Life Science 


e Earth and Space Sciences 


Physical Science deals with matter, energy, and motion; Life Science deals with structures and 
functions of living systems and changes in living systems; and Earth and Space Sciences deal with 
Earth in space and time, Earth structures, and Earth systems. Details about the science content and 


the science content statements can be found in chapter two. 


Science Practices 


The second dimension of the framework is defined by four science practices: 


e Identifying Science Principles 
e Using Science Principles 

e Using Scientific Inquiry 

e Using Technological Design 


These practices can be combined with any science content statement to generate student per- 
formance expectations, and assessment items can then be developed based on these performance 
expectations. The cognitive demands placed on students as they engage in assessment tasks are also 


described. The science practices and cognitive demands are explained more fully in chapter three. 


Distribution of Items 


As measured by student response time, the distribution of items by content area should be as 
follows: approximately equal across Physical Science, Life Science, and Earth and Space Sciences 
at grade 4; more emphasis on Earth and Space Sciences at grade 8; and a shift to more emphasis on 
Physical Science and Life Science at grade 12. With respect to science practices, at all grades the 
greatest emphasis should be on identifying and using science principles and slightly less than a third 
of the time should be spent on items related to using scientific inquiry. Specific recommended per- 


centages are discussed in chapter four. 
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Types of Items 


Item types for the NAEP Science Assessment fall into two broad categories: selected-response 
items (such as multiple choice) and constructed-response items (such as short answer). Starting with 
the 2019 assessment, students engage with all selected-response and constructed-response items 
in a digital platform. As measured by student response time, 50 percent of the assessment items at 
each grade level should be selected response and 50 percent should be constructed response. To 
further probe students’ abilities to combine their understanding with the investigative skills that 
reflect practices, a subset of the students sampled should receive an additional 30 minutes to com- 
plete hands-on performance or interactive computer tasks. At each grade, at least four of these tasks 
should be included. Of these four tasks, there should be at least one hands-on performance task 
and one interactive computer task; the number of interactive computer tasks should not exceed the 
number of hands-on performance tasks. Chapter four contains more information about types of 
items. 


Hands-on Performance Tasks 


In hands-on performance tasks, students manipulate selected physical objects and try to solve a 
scientific problem involving the objects. NAEP hands-on performance tasks should provide stu- 
dents with a concrete task (problem) along with equipment and materials. Students should be given 
the opportunity to determine scientifically justifiable procedures for arriving at a solution. Students’ 
scores should be based on both the solution and the procedures created for carrying out the investi- 


gation. Further discussion about hands-on performance tasks can be found in chapter four. 


Interactive Computer Tasks 


There are four types of interactive computer tasks: (1) information search and analysis, (2) 
empirical investigation, (3) simulation, and (4) concept maps. Information search and analysis items 
pose a scientific problem and ask students to query an information database and analyze relevant 
data to address the problem. Empirical investigation items place hands-on performance tasks on the 
computer and invite students to design and conduct a study to draw conclusions about a problem. 
Simulation items model systems (e.g., food webs) and ask students to manipulate variables, and pre- 
dict and explain resulting changes in the system. Concept map items probe aspects of the structure 
or organization of students’ scientific knowledge by providing concept terms and having students 
create a logical graphic organizer. Chapter four contains more information about interactive com- 
puter tasks. 
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Special Studies 


On occasion, NAEP carries out special studies to inform future assessments. Three special study 


proposals ranked as top priority for the NAEP Science Assessment: 


e “Exchangeability” of hands-on performance and interactive computer investigations 

e Impact of variation in item format and language demand on the performance of English 
language learners and students with disabilities 

e Computer adaptive testing to assess the development of student understanding of Earth 


systems 


Details of these proposed studies are found in chapter four. 


Achievement Levels 


Results of the NAEP Science Assessment are reported as average scores for groups of students 
and as percentages of students who attain the NAEP Basic, NAEP Proficient, or NAEP Advanced 


achievement levels. Descriptions of these achievement levels are contained in appendix B. 
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CHAPTER ONE 


OVERVIEW 


NAEP measures student science achievement nationally, state by state, and, most recently, across 
selected urban school districts. Periodically, the framework underlying the science assessment is 
revised or updated. This document, Science Framework for the 2019 National Assessment of Educa- 
tional Progress, is based on the same framework used for the 2009 NAEP Science Assessment. The 
framework provides guidance on the science content to be assessed, the types of assessment ques- 


tions, and the administration of the assessment. 


For more than 35 years, NAEP has gathered information on student achievement in selected 
academic subjects. Originally, assessments were age-based samples of 9-, 13-, and 17-year-old stu- 
dents. Beginning in 1983, the assessment also has included grade-based samples of students in grades 
4, 8, and 12. Currently, long-term trend NAEP continues to assess 9-, 13-, and 17-year-olds in math- 
ematics and reading, while main NAEP assesses students in grades 4, 8, and 12. More information 
about differences between long-term trend and main NAEP can be found on the Internet at http:// 
nces.ed.gov/nationsreportcard/about/Itt_main_diff-asp (NCES 2005b). 


NAEP has become an important source of information on what US. students know and are able 
to do in reading, mathematics, science, US. history, writing, and other subjects. In addition, NAEP 
provides information on how student performance has changed over time. Since the 1990s, in addi- 
tion to the national-level assessments, NAEP has conducted and reported state-level assessments at 
grades 4 and 8 in reading, mathematics, writing, and science. State-level as well as national science 
assessments were conducted in 1996, 2000, 2005, 2009, 2011, and 2015. The resulting data on stu- 
dent knowledge and performance have been accompanied by background information that allows 
analyses of a number of student demographic and instructional factors related to achievement. 

The assessments have been designed to allow comparisons of student performance over time and 
among subgroups of students according to region, parental education, gender, and race/ethnicity. 

In 2002, NAEP began a Trial Urban District Assessment (TUDA) in districts that volunteered to 
participate. TUDA continued through 2005, when 10 districts took part in NAEP assessments that 


produced district-level results. The TUDA program has grown in size, with 27 districts volunteering 
to participate in 2017. 


Need for a New Framework 


The framework that guided the NAEP Science Assessments administered in 1996, 2000, 2005, 
2009, 2011, and 2015 was developed in the early 1990s. Since then, the following developments 
have taken place, making it necessary to create a new framework for assessing science in 2009 and 


beyond: 


e Publication (for the first time) of National Standards for science literacy in National 
Science Education Standards (National Research Council [NRC] 1996) (National Stan- 
dards) and Benchmarks for Scientific Literacy (American Association for the Advancement 
of Science [AAAS] 1993) (Benchmarks). Since their publication, these two national docu- 
ments have informed state science standards. 

e Advances in science research (e.g., in the relationship between human activity and the 
natural world) that have increased knowledge and, as a consequence, have influenced the 
school curriculum in the fields of Physical, Life, and Earth and Space Sciences. 

e Advances in cognitive research (e.g., in how students learn increasingly complex material 
over time) that have yielded new insights into how and what students learn about science 
(NRC 1999c). 

e Growth in the prevalence of science assessments nationally and internationally, includ- 
ing the requirements in the current federal education legislation (No Child Left Behind) 
for science assessment starting in 2007, the ongoing international assessment (TIMSS) 
(http://timss.bc.edu), and PISA (www.pisa.oecd.org). 

e Growth in innovative assessment approaches that probe students’ understanding of 
science in greater depth than before (e.g., clusters of items tapping students’ conceptions of 
the natural world), sometimes with the use of computer technology (NRC 2001). 

e Increased inclusion of formerly excluded groups in science assessments (e.g., students 
with disabilities and English language learners), requiring a new assessment to be as acces- 
sible as possible and also to incorporate accommodations so the widest possible range of 
students can be fairly assessed. Accommodations should not alter the science constructs 


being measured. 


Context for Planning the Framework 


Any NAEP framework must be guided by NAEP purposes as well as the policies and proce- 
dures of the Governing Board, which oversees NAEP. For the NAEP Science Assessment, the main 
purpose of the framework is to establish what students should know and be able to do in science. 


Meeting this purpose requires a framework built around what communities involved in science and 


science education consider as a rigorous body of science knowledge and skills that are most import- 
ant for NAEP to assess. 


In prioritizing the content, the framework developers used the NAEP Science Assessment 
steering committee guidelines (presented later in this chapter), which recommended the two 
national documents (National Standards and Benchmarks) as representative of the leading science 
communities and their expectations for what students should know and be able to do in science. As 
curriculum frameworks, however, these documents cover a very wide range of science content and 
performance. The inclusive nature of both documents demonstrates the difficulty of identifying 
a key body of knowledge for students to learn in science and, therefore, what should be assessed. 
Neither document limits or prioritizes content as is necessary for developing an assessment, posing 
a considerable challenge to the framework developers. The development of the framework also 
was informed by research in science and science education, best practices, international assessment 


frameworks, and state standards. 


The Framework Development Process 


In September 2004, the Governing Board awarded a contract to WestEd and CCSSO to 
develop a recommended framework and specifications. WestEd and CCSSO, in collaboration with 
AAAS, the Council of State Science Supervisors (CSSS), and the National Science Teachers Asso- 
ciation (NSTA), used a process designed to accomplish the purposes of this project with special 
attention given to the assessment issues specific to K-12 science achievement. The process for devel- 
oping the framework and related products was inclusive and deliberate, designed to achieve as much 


broad-based input as possible. 


A two-tiered committee structure, consisting of a steering committee and a planning commit- 
tee, provided the expertise to develop the framework as specified by the Governing Board. (See 
“NAEP Science Project Staff and Committees” for lists of committee members.) The two commit- 
tees were composed of members who were diverse in terms of role, gender, race/ethnicity, region of 


the country, perspective, and expertise regarding the content of the assessment to be developed. 


The science assessment steering committee included leaders in science, science education, 
general education, assessment, and various public constituencies and set the course for the project. 
Functioning as a policy and oversight body, this group developed a charge that outlined the plan- 
ning committee’s responsibilities in developing the framework. The committee also reviewed and 


provided feedback on drafts of the framework and related materials. 
The science assessment planning committee, supported by the project staff, was the develop- 


ment and production group responsible for drafting the framework, the specifications, recom- 


mendations for background variables, designs for one or more small-scale studies, and preliminary 
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science achievement level descriptions. This committee was made up of science teachers, district and 
state science personnel, science educators in higher education, scientists, and assessment experts. The 
planning committee’s work was guided by policies, goals, and principles identified by the steering 
committee. In addition, the planning committee used a number of resources to facilitate its work. 
These resources included an issues and recommendations paper (Champagne et al. 2004) devel- 
oped specifically for this NAEP project; the frameworks and specifications for the 2005 NAEP 
Mathematics Assessment and 1996-2005 NAEP Science Assessments; other NAEP reports and 
documents produced by the Governing Board and NCES; international assessment frameworks; 
syntheses of state and national curriculum standards; and research papers and resources provided by 


steering and planning committee members and project staff. 


The structure for conducting the work consisted of a series of meetings. From December 2004 
through September 2005, the steering committee met three times and the planning committee met 
six times; two of the steering committee meetings overlapped with planning committee meetings. 
Governing Board staff supported and participated in the work of the committees during the meet- 
ings. In addition, between formal work sessions, Governing Board members and staff provided 


ongoing feedback and guidance on project documents and processes. 


During spring 2005, CCSSO led a series of outreach efforts to solicit feedback on draft versions 


of the framework. Formal activities included the following: 


e A series of 13 regional meetings held across the country and hosted by CCSSO and CSSS 
members 

e A national meeting of CSSS representatives 

e A Web-based survey of science teachers distributed through NSTA 

e An invitational science and industry feedback forum held in Atlanta, Georgia, in conjunc- 


tion with a Governing Board meeting 


These activities are discussed in A Summary of National Feedback Provided on Preliminary Drafts 
Gathered from Surveys and Regional and National Feedback Meetings (CCSSO 2005). Feedback 
from these sessions has been incorporated into the framework. Examples include reducing the num- 
ber of statements of science content to be assessed; comparing the old and new science frameworks; 
and ensuring a high level of consistency in scope, specificity, language, and format across the science 


content areas. 


Other related outreach activities included but were not limited to presentations and sessions 
held with AAAS, the CSSS annual conference; the NSTA national and regional conventions; 
meetings of the NRC’s Board on Science Education and Committee on Science Learning K-8; and 
CCSSO’s Mega-SCASS (State Collaborative on Assessment and Student Standards) conference, 


large-scale assessment conference, and Education Information Management Advisory Consortium 
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(EIMAC). The Governing Board engaged an external review panel to evaluate the draft framework 
and convened a public hearing to gather additional input during the development process. (See 
“NAEP Science Project Staff and Committees” for more complete lists of individuals and organi- 
zations that contributed to the development of this framework.) The planning committee reviewed 
feedback from these groups as well as that from the steering committee and made changes as it 
deemed appropriate. After final approval from the steering committee, the framework, specifica- 
tions, and related products were submitted to the Governing Board for approval. The Governing 
Board unanimously approved the Science Framework on November 18, 2005. Final copies of the 


specifications and related products were submitted to the Governing Board in March 2006. 


Steering Committee Guidelines 


Because the steering committee guidelines were important to the planning committee in 
developing the framework, the major points of the guidelines are summarized below. (The complete 
document is contained in appendix A.) The guidelines consist of criteria that the framework and 


specifications must meet: 


e The framework is informed by the National Standards and Benchmarks. The frame- 
work should reflect the nation’s best thinking about the importance and age-appropri- 
ateness of science principles and thus be informed by two national documents that were 
subject to extensive internal and external reviews during their development. 

e The framework reflects the nature and practice of science. The National Standards and 
Benchmarks include standards that address science as inquiry, the nature of science, history 
of science, and the manmade world. The framework should emphasize the importance of 
these aspects of science education and should include the expectation that students will 
understand the nature and practice of science. Because the scientific disciplines are no 
longer practiced in isolation, and research that cuts across discipline boundaries is com- 
mon, the framework should identify some of the science concepts and skills that cut across 
the assessed content areas. The framework should address science in both the natural and 
manmade world, as well as social and historical contexts. 

e The framework uses assessment content, formats, and accommodations consistent 
with the objectives being assessed. It should be guided by the best available research on 
assessment item design and delivery. The framework should include student diversity as 
reflected in gender, geographic location, language proficiency, race/ethnicity, socioeco- 
nomic status, and disability. The assessment should be designed and written to be accessi- 
ble by the majority of students, minimizing the need for special accommodations for both 
students with disabilities and English language learners. However, students with special 
needs should be provided appropriate accommodations to allow them to participate in the 
assessment. The framework should reflect knowledge about the acquisition of key science 


concepts over time, based on research about how students learn. Critical content and skills 
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should be articulated and assessed across grades 4, 8, and 12 (vertically), as well as across 
the fields of science (horizontally) by creating items that are deliberately layered to achieve 
these goals. 

e The framework uses a variety of assessment formats. This includes well-constructed 
multiple-choice and open-ended items as well as performance tasks. In addition, multiple 
methods of assessment delivery should be considered, including appropriate uses of com- 
puter technology. 

e Each achievement level—NAEP Basic, NAEP Proficient, and NAEP Advanced— 
includes a range of items assessing various levels of cognitive knowledge that is broad 
enough to ensure each knowledge level is measured with the same degree of accuracy. 
Descriptions of NAEP Basic, NAEP Proficient, and NAEP Advanced must be as clear as 
possible. 

e Connections among the framework, the specifications, and the assessment items 
themselves are transparent, coherent, and have a consistent level of specificity. The 
specifications should be written with detail consistent with the framework. The content 
addressed in the specifications should reflect the standards and focus on the significant 
information and knowledge that students should retain over time (e.g., big ideas, funda- 
mental understandings). The verbs used in the specifications should describe the expected 
target for assessment (e.g., identify, describe, evaluate, relate, analyze, and demonstrate). 
The content expectations should match in level of specificity and scope across the disci- 
plines. The specifications should follow the idea of learning progressions. To assess over- 
arching concepts or themes, the specifications should reflect a layered understanding of 
growth in knowledge of the concepts. 

e The framework addresses the use of assessment data to conduct research on science 
learning and to improve science achievement. Data (background variables) from the 
assessment should be collected in a way that provides details about the characteristics of 
the students being assessed (e.g., race/ethnicity and gender), the academic preparation of 
their teachers, and the nature of their schools. Such data provide feedback to educators for 


improving science instruction and learning. 


Uses of NAEP Data 


For more than four decades, NAEP has provided information integral to reporting on the 
condition and progress of education at grades 4, 8, and 12 for the nation and, more recently, for the 
states and for a set of large, urban school districts. Legislation concerning NAEP states that its pur- 
pose is to provide, in a timely manner, a fair and accurate measurement of student academic achieve- 
ment and reporting of trends in such achievement in reading, mathematics, and other subject matter 
(Public Law 107-279). 


Because of its rigorous design and methodology, NAEP reports are increasingly used for mon- 
itoring the state of education in the subjects that are assessed, as models for designing other large- 


scale assessments, and for secondary research purposes. 


Monitoring 


As the nation’s only ongoing survey of students’ educational progress, NAEP has become an 
increasingly important resource for obtaining information on what students know and can do. 
Because the information it generates is available to policymakers, educators, and the public, NAEP 
can be used as a tool for monitoring student achievement in reading, mathematics, science, and 
other subjects at the national, state, and selected district levels. For example, NAEP reports (known 
as The Nation’s Report Card) compare student performance in a given subject across states, within 
the subject over time, or among groups of students within the same grade. NAEP also reports 
long-term achievement trends for 9-, 13-, and 17-year-olds in reading and mathematics (e.g., Perie, 
Moran, and Lutkus 2005). To the extent that individual state standards reflect the common core 
of knowledge and skills specified in the framework, state comparisons can legitimately be made. If 
a state has unique standards, any comparison is limited by the degree of mismatch between NAEP 
content and state content. Even with this caveat, NAEP still stands as a key indicator of what stu- 


dents know and can do in science at grades 4, 8, and 12. 


Model of Assessment Development and Methods 


NAEP assessment frameworks and specifications documents are used as resources for interna- 
tional, state, and local curriculums and assessments. The broad-based process used in the develop- 
ment of the frameworks and specifications means that current thinking and research are reflected 
in the descriptions of what students should know and be able to do in a given subject. In addition, 
NAEP uses a rigorous and carefully designed process in developing the assessment instruments. 
Pilot tests and internal and external reviews ensure that NAEP assessments are reliable and valid 
with respect to what they attempt to accomplish. This sophisticated methodology serves as a model 
for other assessment developers. Given the No Child Left Behind requirement to assess students in 


science, states may wish to use NAEP as one model to guide their own assessment development. 


Research and Policy 


NAEP data include subject-matter achievement results (reported as both scale scores and 
achievement levels) for various subgroups; background information about schools, teachers, and 
students at the subgroup level (e.g., course-taking patterns of Hispanic male 12th graders); state- 
level results; reports for a set of large urban districts; history of state and district participation; and 


publicly released assessment questions, student responses, and scoring guides. The NAEP website 
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(http://nces.ed.gov/nationsreportcard) contains user-friendly data analysis software to enable pol- 
icymakers, researchers, and others to examine all aspects of NAEP data, perform significance tests, 
and create customized graphic displays of NAEP results. These data and software tools can be used 


to inform policymaking and for secondary analyses and other research purposes. 


Challenges of Developing a NAEP Assessment 


There are three major challenges in developing a NAEP framework: (1) measurement con- 
straints and the nature of the items included on the assessment, (2) time and resource constraints 
and how much can be assessed in NAEP, and (3) the timeline for the framework and the difficulty 
of developing a 10-year framework with the rapid explosion of knowledge in the Information Age. 
Each of these challenges is discussed below. 


Measurement Constraints 


Like any large-scale assessment in education, the workplace, or clinical practice, NAEP is 
constrained in what it can measure. This has implications for the proper interpretation of NAEP 
Science Assessment results. The NAEP Science Framework is an assessment framework, not a 
curriculum framework. Although the two are clearly interrelated, each has a different purpose and a 
different set of underlying assumptions. A curriculum framework is designed to inform instruction, 
to guide what is taught, and often, to guide how it is taught. It represents a very wide universe of 
learning outcomes from which teachers pick and choose what and how they teach. An assessment 
framework is a subset of the achievement universe from which assessment developers must choose 
to develop sets of items that can be assessed within time and resource constraints. Hence, the science 
content to be assessed by NAEP has been identified as that considered central to the Physical, Life, 
and Earth and Space Sciences. As a result, some important outcomes of science education that are 
difficult and time consuming to measure (such as habits of mind, sustained inquiry, and collabora- 
tive research), but valued by scientists, science educators, and the business community, will be only 
partially represented in the framework and in the NAEP Science Assessment. Moreover, the wide 
range of science standards in the guiding national documents that could be incorporated into the 
framework had to be reduced in number so as to allow some in-depth probing of fundamental sci- 
ence content. Asa result, the framework and the specifications represent a careful distillation that is 
not a complete representation of the original universe of achievement outcomes desirable for science 


education. 
Time and Resource Constraints 


What NAEP can assess is limited by time and resources. Like most standardized assessments, 


NAEP is an “on-demand” assessment. It ascertains what students know and can do in a limited 
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amount of time (60 minutes for questions in the standard digital platform and, for a subset of stu- 
dents sampled, an additional 30 minutes for hands-on performance or interactive computer tasks) 
and with limited access to resources (e.g., reference materials, feedback from peers and teachers, 
opportunities for reflection and revision). The national and state standards, however, contain goals 
that require extended time (days, weeks, or months). Therefore, to assess student achievement in the 
kinds of extended activities that are a central feature of the national and state standards and many 


science curricula, it would be necessary to know (for example) the quality of students’: 


e reasoning while framing their research questions; 

e planning for data collection and the execution of the plan; 

e abilities to meet unpredictable challenges that arise during an actual, ongoing scientific 
investigation; 

e lines of argument in deciding how to alter their experimental approach in light of new 
evidence; 

e engagement with fellow students and/or the teacher in interpreting an observation or 
result and deciding what to do about it; and 

e deliberations and reasoning when settling on the defensible conclusions that might be 


drawn from their work. 


Like other on-demand assessments, NAEP cannot be used to draw conclusions about student 
achievement with respect to the full range of goals of science education. States, districts, schools, 
and teachers can supplement NAEP and other standardized assessments to assess the full range of 
science education standards. In addition to describing the content and format of an examination, 
assessment frameworks like this one signal to the public and to teachers the elements of a subject 
that are important. The absence of extended inquiry in NAEP, however, is not intended to signal its 
relative importance in the curriculum. Indeed, because of the significance of inquiry in science edu- 
cation, the framework promotes as much consideration of inquiry as can be accomplished within 


the time and resources available for assessment. 


Balancing Current and Future Curricula 


The framework attempts to strike a balance between what can reasonably be predicted about 
future school science and what students are likely to encounter in their curriculum and instruction 
now and in the near future. It is a significant challenge to write a framework for the future. Cut- 
ting-edge science research creates new knowledge at the intersection of disciplinary boundaries. For 
example, research on human and natural systems has generated new understanding about environ- 
mental science that is closely linked to knowledge generated in the Physical, Life, and Earth and 
Space Sciences. Although the framework is organized into the more traditional Physical, Life, and 


Earth and Space Sciences, features of current science research are woven throughout. 
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Another example of burgeoning knowledge relates to technology and technological design and 
the role of both in the NAEP Science Assessment. Technology and technological design are closely 
interrelated with science, yet the focus of this assessment is on science. Hence, technology and tech- 
nological design are included in the framework but are limited to that which has a direct bearing on 
the assessment of students’ science achievement (see chapter three). 


The framework is intended to be both forward looking (in terms of the science content that will 
be of central importance in the future) and reflective (in terms of current school science). Because it 
is impossible to predict with certainty the shape of school science, the choices made for this frame- 


work should be revisited in response to future developments in school science. 


Achievement Levels 


Public Law 107-279 specifies the Governing Board’s responsibilities regarding NAEP, includ- 
ing the identification of appropriate achievement goals for each age and grade in the subject areas 
assessed by NAEP. 


To carry out its mandated responsibility to set appropriate achievement goals for NAEP, the 
Governing Board adopted an achievement levels policy in 1989 (last modified in 2018). This policy 
establishes three levels of achievement—NAEP Basic, NAEP Proficient, and NAEP Advanced. 
NAEP Basic denotes partial mastery of prerequisite knowledge and skills that are fundamental for 
proficient work at each grade. NAEP Proficient represents solid academic performance for each 
grade assessed. Students reaching this level have demonstrated competency over challenging subject 
matter, including subject-matter knowledge, application of such knowledge to real-world situations, 
and analytical skills appropriate to the subject matter. NAEP Advanced signifies superior perfor- 
mance. These levels are the primary means of reporting NAEP results to the general public and 
policymakers regarding what students should know and be able to do on NAEP assessments. (See 
appendix B for the NAEP science achievement level descriptions and additional information about 


their development and use.) 


Introduction to the Assessment Framework 


Science comprises both content and practices. The NAEP Science Assessment provides a 
snapshot view of what the nation’s 4th, 8th, and 12th graders know and can do in science. One 
expects students, as a result of their education and life experiences, to have learned about the prin- 
ciples (along with the facts, concepts, laws, and theories) that have been verified by the commu- 
nity of scientists, as well as how scientists discover regularities in the natural world.' NAEP will 


assess students’ abilities to identify and use science principles as well as use scientific inquiry and 


“Science principles” is used throughout this framework to denote not only the principles but also the facts, concepts, theories, and laws of science. 
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technological design (see chapter three). Although the framework distinguishes content from prac- 


tice, the two are closely linked in assessment as in science itself. 


The framework addresses scientific knowledge and processes. Science is a way of knowing about 
the natural world based on tested explanations supported by accumulated empirical evidence. Expla- 
nations of natural phenomena that rely on nonscientific views are not reflected in the framework. 
The committees responsible for the framework development relied on National Standards, Bench- 
marks, international frameworks, and state standards for content about the nature and practice of 


science. As stated in the National Standards: 


Scientific explanations must meet certain criteria. First and foremost, they must be con- 
sistent with experimental and observational evidence about nature, and must make accurate 
predictions, when appropriate, about systems being studied. They should also be logical, 
respect the rules of evidence, be open to criticism, report methods and procedures, and 


make knowledge public (p. 201). 


The design of the NAEP Science Assessment is guided by the framework’s descriptions of the 
science content and practices to be assessed. Exhibit 1 illustrates how content and practices are 
combined (crossed) to generate performance expectations. The columns contain the science content 
(defined by content statements—propositions that express science principles—in three broad areas) 
and the rows contain the four science practices. A double dashed line distinguishes identifying 
science principles and using science principles from using scientific inquiry and using technolog- 
ical design. The first two practices can be generally considered as “knowing science” and the last 
two practices can be considered as the application of that knowledge to “doing science” and “using 
science to solve real-world problems.” The cells at the intersection of content (columns) and prac- 
tices (rows) contain student performance expectations. Because content and practice categories are 


not entirely distinct, some overlap in the resultant performance expectations is to be expected (as 


denoted by dashed lines). 
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Exhibit 1. Crossing content and practices to generate performance expectations 


Science Content 


: : ; : Earth and Space 
Physical Science Life Science pe 
Sciences 
Content Statements Content Statements 


Content Statements 


Identifying Performance | Performance Performance 
Science Principles Expectations Expectations Expectations 

PE Gas eS ee ee aia ale fice eye ieee yee foe Stee ee eG 

8 | Using Science Performance Performance Performance 

om 1 1 1 

~~ . . 1 . 1 . 1 . 

3 Principles Expectations Expectations Expectations 

a 4 i i 

OQ f---- o-oo ror - oe eee er ere ee Peet esse eee ee eee ete see eee Peer Sete osteo eee ee See eee ses 

i) : : i i 

& | Using Performance Performance Performance 

om . . . q . 1 . 1 . 

o Scientific Inquiry Expectations Expectations Expectations 
Using Performance Performance Performance 
Technological Design ' Expectations Expectations Expectations 


Exhibit 2 illustrates the fuller process of generating assessment items and interpreting student 
responses. An item is an individual question or exercise on the NAEP Science Assessment and is 
used to gather information about students’ knowledge and abilities. Exhibit 2 begins with student 
performance expectations, which describe in observable terms what students are expected to know 
and do on the assessment. These performance expectations guide the development of assessment 
items. The cognitive demands (see chapter three) of the items can then be used to interpret students’ 
responses as evidence of what students know and can do in science (see Ruiz-Primo et al. 2001 for 
more information on cognitive interpretations of performances on assessment tasks). Exhibit 2 
suggests a linear process, but the development of an assessment is iterative (e.g., assessment items are 


modified based on student responses provided on trials of pilot versions). 
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Exhibit 2. Generating items and interpreting responses 


Performance 


Expectations 


gener ate 


Assessment 
* J 


tems 


( “ea 


Items place cognitive 
demands on students. These 
cognitive demands can be 


that elicit 


Student 
Responses 


used as tools for design- 


ing items and interpreting that ate interpreted 


student responses. 


L a. 


as evidence of 


“>a! What Students 
Know and Can 
Do in Science 


Overview of Framework Chapters 


This section provides an overview of the content in each chapter of the framework. For ease of 
understanding, three types of textboxes are used throughout the framework: clarification, crosscut- 
ting content, and illustrative item. Clarification textboxes provide details on potentially confusing 
topics, such as the distinction between mass and weight. Crosscutting content textboxes provide 
descriptions of science content that cuts across the three broad content areas, such as biogeochem- 
ical cycles. Illustrative item textboxes provide assessment items that exemplify recommendations 
discussed in the text. 


Chapter Two: Science Content 


Key principles as well as facts, concepts, laws, and theories that describe regularities in the nat- 
ural world are presented in chapter two as a series of content statements to be assessed at grades 4, 
8, and 12. Taken together, these statements comprise the NAEP science content. They define only 
what is to be assessed by NAEP and are not intended to serve as a science curriculum framework. 
The content statements should be revisited periodically in response to new developments in science 


research and school curriculum. 
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The content statements are organized according to the three broad content areas that generally 


comprise the K-12 school curriculum: 


e Physical Science 
e Life Science 


e Earth and Space Sciences 


Classifying content statements into one content area is not always clear-cut, but doing so helps 
ensure the areas of science are assessed in a balanced way. Some common content is found to be sig- 
nificant in each of the three content areas (e.g., energy conservation and its associated principles are 
applicable to the living and nonliving systems studied by Physical, Life, and Earth scientists). This 


crosscutting content is further described in chapter two. 


The content statements are derived from National Standards and Benchmarks and are informed 
by international frameworks and state standards. Content statements have been added where 
warranted by advances in science since the development of the standards documents. The selection 
of science content statements to be assessed at each grade level focuses on principles central to each 
discipline, tracks related ideas across grade levels, and limits the breadth of science knowledge to be 
assessed. The selection of content statements used an iterative approach and took into account the 


many perspectives of various stakeholders. 


Chapter Three: Science Practices 


The following science practices are found in most or all of the major sources used to develop the 
framework (see standards documents listed above). The practices to be assessed at grades 4, 8, and 12 


are organized into four categories: 


Identifying Science Principles 


Using Science Principles 


Using Scientific Inquiry 


Using Technological Design 


As shown in exhibit 1, the Physical Science, Life Science, and Earth and Space Sciences content 
statements listed in chapter two can be deliberately combined (crossed) with each of the above four 
practices to generate specific performance expectations. These performance expectations are written 
in observable terms and guide the development of assessment items. Performance expectations thus 
provide an outline of what the NAEP Science Assessment will expect as evidence of what students 
know and can do. At the end of chapter three, examples of performance expectations, items, and 


interpretations of student responses are provided. (See the specifications for more examples.) 
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Students’ understanding increases over time as they learn more, moving from initially naive to 
increasingly more sophisticated knowledge about the natural world. These learning progressions are 
further described in chapter three. 


Chapter Four: Overview of the Assessment Design 


Chapter four provides an overview of the assessment design, including a discussion of assess- 
ment item contexts, types of items, distribution of items, accessibility concerns for English language 


learners and students with disabilities, and recommendations for small-scale special studies. 


Beyond the science content statements and practices, there are other valuable components of 
science that NAEP will not directly assess. These components—the history and nature of science 
and the relationship between science and technology—are treated as possible contexts in which 


assessment items may be presented. 


The assessment includes selected-response (multiple-choice) items and constructed-response 
items (which include short and extended constructed-response items as well as concept-mapping 
tasks). Some combination items may require more than one response. They include item clusters, 
predict-observe-explain (POE) item sets, hands-on performance tasks, and interactive computer 
tasks. The responses requested may be all selected response, all constructed response, or a mixture of 
both. Starting with the 2019 assessment, students engage with all selected-response and construct- 
ed-response items in a digital platform. (See chapter four for a fuller explanation of types of items, 


including examples.) 


Chapter four recommends three types of percentages for item distribution (as measured by 


student response time) at each grade level: 


e Items by content area (Physical, Life, and Earth and Space Sciences) 
e Items by science practice (Identifying Science Principles, Using Science Principles, Using 
Scientific Inquiry, and Using Technological Design) 


e Items by type (selected-response and constructed-response items) 


Additional specifications about the number of hands-on performance tasks and interactive com- 


puter tasks are also provided. 


Chapter four also describes considerations for English language learners and students with dis- 
abilities. In particular, NAEP assessments need to be responsive to growing demands for increased 
inclusion of all types of students in the general curriculum and increased emphasis and commitment 
to serve and be accountable for all students. A number of small-scale special studies also are recom- 


mended at the end of chapter four. 
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Comparison of 1996-2005 NAEP Science Framework 
and 2009-2019 NAEP Science Framework 


The chapter overviews provided above reflect the differences between the NAEP 1996-2005 


science framework and the framework to be used for the 2009, 2011, 2015 and 2019 NAEP science 


assessments. The assessments resulting from this framework start a new NAEP science trend (i.e., 


measure of student progress in science). One difference between the two frameworks is that the 


2009 steering and planning committees drew on a variety of new standards and assessments. They 


also were able to extract findings from research in science, science education, and cognition and to 


consider the use of technology to increase the options for assessment administration. Exhibit 3 lists 
the major differences between the 1996-2005 and 2009-2019 NAEP science frameworks. 


Exhibit 3. Comparison of NAEP science frameworks: 1996-2005 and 2009-2019 


1996-2005 Framework 


2009-2019 Framework 


Science Content 


Few science standards available on which to base 
the content to be assessed 


Content drawn from existing standards and 
assessment frameworks: National Standards, 
Benchmarks, TIMSS, PISA, and state standards 


Content areas organized into Physical Science, 
Life Science, and Earth Science 


Content areas organized into Physical Science, 
Life Science, and Earth and Space Sciences 


Recommendations on distribution of questions 
by fields of science and grade: 

approximately equal distribution in grades 4 and 
12; a somewhat heavier emphasis on Life Science 


in grade 8 


Recommendations on distribution of questions 
by science content area and grade: equal weight 
for all three sciences in grade 4; emphasis on 
Earth and Space Sciences in grade 8; emphasis 
on Physical Science and Life Science at grade 12 
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Exhibit 3 (continued). Comparison of NAEP science frameworks: 
1996-2005 and 2009-2019 


1996-2005 Framework 


2009-2019 Framework 


Content presented as bullets and short phrases 


Content presented as statements in tables orga- 
nized by science content subtopics (e.g., forces 
affecting motion from Physical Science) and by 


iS 
5 grade level 
=| 
3 Framework employed three abstract themes: 
g systems, models, and patterns of change 
Vv Fe 
3 Framework employs crosscutting content among 
a ; ; : 
Life, Physical, and Earth and Space Sciences 
Assessment asked questions about the nature 
of science 
Knowing and doing dimension organized into Science practices dimension organized into 
conceptual understanding, scientific investiga- Identifying Science Principles, Using Science 
tion, and practical reasoning Principles, Using Scientific Inquiry, and Using 
Technological Design 
Nature of science treated within science prac- 
tices, particularly Using Science Principles and 
Using Scientific Inquiry 
Science practices assessed were largely Science practices assessed take into account 
experience based extant research and the cognitive complexity of 
& the items 
a Assessment included items on practical reason- 
(aE, . . . . . 
ea ing (i.e., applying science to suggest effective 
& | solutions to everyday problems) 
Vv 
= 
2) ‘, é . 
a Assessment includes questions on technological 


design (i.e., the systematic process of applying 


science knowledge and skills to solve problems in 
a real-world context) 


Forty-five percent of the assessment focused on 
conceptual understanding 


Sixty percent of the assessment focuses on con- 
ceptual understanding (Identifying and Using 
Science Principles) 


Learning progressions (i.e., connected sequences 
of science performances across grade spans) are 


included 
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Exhibit 3 (continued). Comparison of NAEP science frameworks: 
1996-2005 and 2009-2019 


1996-2005 Framework 


2009-2019 Framework 


Assessment Items 


Assessment uses the history of science and the 
relationship between science and technology as 
contexts for questions 


Assessment included both paper-and-pencil and 
hands-on performance tasks 


Assessment includes all digitally based questions 
starting in 2019, as well as hands-on perfor- 
mance tasks and interactive computer tasks 


No illustrative items to convey science knowl- 
edge or practices in the framework; only a 
few suggested ideas for items provided in the 
specifications 


Illustrative items that convey science knowledge 
and practices are included in both the frame- 
work and the specifications 


Framework and specifications include guide- 
lines for assessing students with disabilities and 
English language learners 


Framework includes examples showing how 


questions are generated and interpreted 


Students’ naive conceptions about science princi- 
ples are explicitly assessed 
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CHAPTER TWO 


SCIENCE CONTENT 


This chapter presents a series of statements that describe the science content of the NAEP 
Science Assessment. The content statements contain key science principles for NAEP assessment. 
In the framework, the phrase “science principles” is broadly conceived and encompasses not only 
the key principles but also the facts, concepts, laws, and theories of science. To specify the science 
that should be assessed at each grade level, the framework organizes the science content into the 
three broad content areas that generally make up the K-12 school curriculum to which students are 


exposed: 


e Physical Science 
e Life Science 


e Earth and Space Sciences 


Classifying statements into one primary content area is not always clear-cut and is artificial to 
some extent. For example, physicist Ernest Rutherford’s discovery of the nucleus earned him the 
Nobel Prize in Chemistry and physicist Rosalyn Yalow’s work on radioimmunoassay earned her the 
Nobel Prize in Medicine. However, using three broad content areas as an organizer helps ensure that 


key science content is assessed in a balanced way. 


For clarity, exhibits are used to depict the content statements at each grade level. The content 
statements are based on the assumption that a person literate in science is one who understands 
key science ideas, is aware that science and technology are interdependent human enterprises with 
strengths and limitations, is familiar with the natural world and recognizes both its diversity and 
unity, and uses scientific knowledge and ways of thinking for individual and social purposes (see 


AAAS 1994, p. xvii). 


Two types of textboxes are used throughout this chapter. Clarification textboxes provide details 
on potentially confusing content, such as the distinction between mass and weight. Crosscutting 
content textboxes provide descriptions of science content that cuts across the three broad content 


areas, such as biogeochemical cycles. 
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Development of the Content Statements 


The selection and generation of specific content statements at each grade level followed a similar 


approach across the three broad content areas: 


e The National Standards and Benchmarks were used as key documents for identifying the 
science content to be assessed, pursuant to the charge from the steering committee. Various 
tools, primarily crosswalks between the National Standards and Benchmarks (AAAS 1997; 
Kendall and Marzano 2004), were used to cross-check the documents’ content standards 
and benchmark statements; those that were common to both documents were generally 
given priority. On a case-by-case basis, content not represented in both documents was dis- 
cussed and decisions made about inclusion or exclusion. Additions were made where war- 
ranted by scientific advances in the decade or more since the development of the National 
Standards and Benchmarks, or as a consequence of international assessment results from 
TIMSS and PISA. Some of this framework’s content statements are verbatim reproduc- 
tions of statements from the National Standards and Benchmarks. 

e The focus in the selection process was on the central principles of each discipline. The 
content statements in the framework represent foundational and pervasive knowledge, 
key points of scientific theories, and underpinnings upon which complex understanding is 
built; and/or they demonstrate connectivity to other central content. 

e A primary consideration was the grade-level appropriateness and accuracy within grade 
level of content statements. 

e Once key content was identified within subtopics, the progression of ideas and perfor- 
mances, informed by available research, was tracked through grades 4, 8, and 12. 

e A deliberate attempt was made to limit the breadth of science content to be assessed so that 
some important topics could be measured in-depth. Once core content was identified in 
each science area, additional content statements could be added only if others previously 


included were eliminated. 


The selection and generation of content statements for inclusion in the framework was not a 
linear process. Although the planning committee attempted to use clear and concise language, the 
complexities associated with the task of defining what students should know and be able to do in 
science at particular points in their development necessitated an iterative approach that included 
many perspectives. In addition to internal review by the project committees and staff, outreach 
activities gathered external feedback on the content statements from a variety of stakeholders (e.g., 
teachers, school and district administrators, state science education personnel, policymakers, sci- 


entists, and members of the business, industry, and postsecondary communities). The framework 


> Statements are reprinted from National Science Education Standards (NRC 1996) with permission from the National Academy of Sciences, courtesy 
of the National Academies Press, Washington, D.C. Statements are reprinted from Benchmarks for Science Literacy (AAAS 1993) with permission 
from Project 2061, on behalf of the American Association for the Advancement of Science, Washington, D.C. 
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should be revisited in the future as new research becomes available and as the influence of new 


developments in science takes shape in the K-12 curriculum. 


Organization of Science Content 


As described above, this framework organizes science content into three broad content areas 
(Physical Science, Life Science, and Earth and Space Sciences). The content is further organized 
into topics (such as motion), subtopics (such as forces affecting motion), and grade-specific content 
statements. The description of each broad content area follows this structure of increasing specificity 
and is presented in two ways: narrative introductions and content statements presented in exhibits 
(see exhibits 8, 10, and 12 in the sections for Physical Science, Life Science, and Earth and Space Sci- 


ences, respectively). Exhibit 4 summarizes the NAEP science content topics and subtopics. 


Exhibit 4. NAEP science content topics and subtopics 


Physical Science Life Science Earth and Space Sciences 
Matter Structures and Functions Earth in Space and Time 
e Properties of matter of Living Systems e Objects in the universe 
e Changes in matter e Organization and e History of Earth 
Energy development Earth Structures 


e Forms of energy e Matter and energy e Properties of Earth materials 
e Energy transfer and transformations e ‘Tectonics 
conseaon e Interdependence ‘ieaiGateas 


Motion Changes in Living Systems e Energy in Earth systems 


e Heredity and reproduction 


e Motion at the macroscopic e Climate and weather 


e Evolution and diversity 


level e Biogeochemical cycles 


e Forces affecting motion 


As an organizational tool in exhibits 8, 10, and 12, each content statement is preceded by a spe- 
cific code in bold (e.g., “L12.10: Sorting and recombination of genes in sexual reproduction results 
in a great variety of possible gene combinations from the offspring of any two parents”). Within a 
code, the letter denotes broad content area (“P” for Physical Science, “L” for Life Science, and “E” 
for Earth and Space Sciences); the number before the period denotes grade level (grade 4, 8, or 12); 
and the number following the period denotes the content statement’s order of appearance within 
a given content area and grade. Thus, L12.10 denotes that this is the tenth content statement to 
appear in the grade 12 section of the Life Science content statements table. Because the numbering 
within each content area and grade is strictly sequential, code numbers do not necessarily indicate 
any relationships across grades (see, for example, P4.13, P8.13, and P12.13). 
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Interpretation of the Content Statements 


In the framework, the content statements generally follow a form that is consistent with the 


National Standards, Benchmarks, and the practice of the scientific community. The content state- 


ments are phrased as propositions that express science principles. Based on evidence, these principles 


have been verified by the scientific community and are under constant review. Exhibit 5 is an exam- 


ple of how one grade 8 Physical Science principle is represented in the framework, National Stan- 


dards, and Benchmarks. 


Exhibit 5. One grade 8 Physical Science principle represented 
in the framework, National Standards, and Benchmarks 


Framework 


National Standards 


Benchmarks 


P8.16: Forces have magnitude 
and direction. Forces can be 
added. The net force on an 
object is the sum of all the forces 
acting on the object. A nonzero 
net force on an object changes 
the object’s motion; that is, the 
object’s speed and/or direction 
of motion changes. A net force 
of zero on an object does not 
change the object’s motion; that 
is, the object remains at rest or 
continues to move at a constant 


speed in a straight line (p. 35).* 


If more than one force acts on an 
object along a straight line, then 
the forces will reinforce or cancel 
one another, depending on their 
directions and magnitude. 
Unbalanced forces will cause 
changes in the speed or direction 
of an object’s motion (p. 154). 


An unbalanced force acting on 
an object changes its speed or 
direction of motion, or both. If 
the force acts toward a single cen- 
ter, the object’s path may curve 
into an orbit around the center 


(p. 90). 


* This content statement is longer in the framework than in the Benchmarks or National Standards—not 


because it introduces additional science principles, but because it has adopted more detailed language. 


The content statements form the basis for explaining or predicting naturally occurring phenom- 


ena. For example, the above content statement about objects in motion can be used to explain and 


predict the motions of many different specific objects (e.g., an ice skater, an automobile, an electron, 


ora planet). 


The content statements do not include observations of phenomena. As the content statements 


are written, the empirical foundations of the science principles they represent are not detailed. 


Instead, knowledge is presented in general terms, such as patterns in observations or theoretical 


models that account for these patterns. The NAEP assessment will require students to apply content 


statements to specific observations and examples of phenomena. Therefore, the specifications pro- 


vide boundaries on the range of phenomena and examples that are appropriate for inclusion on the 
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NAEP Science Assessment. The boundaries also describe the appropriate instrumentation, represen- 


tations, and technical vocabulary to be included in the assessment. 


The process of item development and making inferences about students’ knowledge and abilities 
(see exhibits 1 and 2) may necessitate further clarification of the content statements themselves. For 
example, this may involve “detailing” the meanings of individual content statements (e.g., “boiling 
point” assumes standard atmospheric pressure) or further defining the boundaries of the content to 
be assessed (e.g., grade 12 students are expected to know that DNA provides instructions for assem- 
bling proteins but not the details of DNA transcription and translation). Exhibit 6 is an example of 
commentary on a grade 8 Physical Science content statement. Although the commentary describes 
key ideas about waves relevant through grade 12, only three of these key ideas are recommended 
for assessment at grade 8. Whenever possible, commentary on content statements was informed 
by available research on learning and cognition. Commentary is intended as helpful notes to item 
writers and is provided for a number of content statements in the Specifications. Commentary is not 
always of the same kind, and the examples provided do not exhaustively cover the content state- 
ments. Assessment developers should continue the process of “detailing” the grade-appropriate prin- 


ciples to be assessed for all content statements sampled for a particular NAEP Science Assessment. 


Exhibit 6. Commentary ona Physical Science content statement 


Grade 8: Energy—Forms of Energy 


Content Statement 


P8.10: Energy is transferred from place to place. Light energy from the Sun travels through space to Earth 
(radiation). Thermal energy travels from a flame through the metal of a cooking pan to the water in the 
pan (conduction). Air warmed by a fireplace moves around a room (convection). Waves—including sound 
and seismic waves, waves on water, and light waves—have energy and transfer energy when they interact 
with matter. 


Commentary 


Wave principles recommended for assessment at grade 8: 


e Waves involve transfer of energy without a transfer of matter. 
e Waves are caused by disturbances and are also themselves disturbances. Some of the energy of these 
disturbances is transmitted by the wave. 


e Water, sound, and seismic waves transfer energy through a material. 
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Exhibit 6 (continued). Commentary ona Physical Science content statement 


Grade 8: Energy—Forms of Energy 


Wave principles that are related but not recommended for assessment at grade 8: 


Some waves are transverse (water, seismic) and other waves are longitudinal (sound, seismic). 
e In transverse waves, the direction of the motion is perpendicular to the disturbance. 
e In longitudinal waves, the direction of motion is parallel to the disturbance. 


Waves (e.g., light waves) traveling from one material to another undergo transmission, reflection, and/ 

or changes in speed. 

e Waves can be described by their wavelength, amplitude, frequency, and speed (speed is frequency mul- 
tiplied by wavelength; energy is a function of the amplitude for nonelectromagnetic waves). 

e Light has dual wave-particle properties. 


Energy and refraction calculations are not recommended for assessment at grade 8. A quantitative under- 


standing of electromagnetic waves is expected at grade 12. See P12.10. 


To fully understand the content statements and their intent, readers of the framework should be 


cognizant of the following: 


e Although all content statements have been assigned a primary classification, some are likely 
to fall into more than one content area. 

e Some assessment items may draw on more than one content statement at a time. 

e Empty cells in the content statement tables denote that a particular subtopic is not recom- 
mended for assessment at that grade level. 

e Retention of foundational knowledge from one grade to the next is assumed; however, if 
the relevant content statement does not appear in a succeeding grade level, it should not be 
assessed. 

e The content statements listed in the framework describe the whole of what is to be assessed 
on the NAEP Science Assessment. The content statements should not be interpreted as a 


complete description of the recommended school science curriculum. 


Crosscutting Content 


Scientists define their specializations narrowly (e.g., astronomy, molecular biology, organic 
chemistry) to organize their research communities; the categories of Physical Science, Life Science, 
and Earth and Space Sciences are helpful for organizing school science. These categorizations mask 
the fact that some science principles cut across the content areas. In this framework, crosscutting 
content is not represented by abstractions such as “models, “constancy and change,’ or “form and 
function,” but is anchored in the content statements themselves. Some examples of crosscutting 
content are described in textboxes that appear throughout the content area introductions: Energy 
Sources and Transfer in Physical Science; Uses, Transformations, and Conservation of Energy in 


Life Science; and Biogeochemical Cycles in Earth and Space Sciences. Additional examples are 
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suggested in the specifications. For instance, the theory of plate tectonics and the evolution of 
Earth’s surface are inextricably linked with environmental pressures (such as geographic barriers), 
speciation, and the evolution of life. Such examples illustrate opportunities for assessing specific 


content in greater depth. 


Physical Science 


Physical Science principles, including fundamental ideas about matter, energy, and motion 
are powerful conceptual tools for making sense of phenomena in physical, living, Earth, and space 
systems. Familiar changes—an ice cube melting, a baseball changing direction after being struck by 
a bat, the appearance of a bolt of lightning, the formation and erosion of mountains, and the growth 


of a plant—can be explained using these fundamental ideas.’ 


Energy is the constant in an ever-changing world. Energy from the Sun fuels electrical storms, 
hurricanes, tornados, and photosynthesis. In turn, the products of photosynthesis (carbohydrates 
and oxygen) react during respiration to fuel life processes, such as growth and reproduction of plants 
and animals. Consequently, it is important for students to develop an understanding of Physical 
Science principles early and to appreciate their usefulness across Physical Science, Life Science, and 


Earth and Space Sciences. 


The Physical Science principles to be assessed are divided into three topics—matter, energy, 
and motion. Matter is the “stuff” of the natural world. Energy is involved in all changes in matter. 
Motion of the heavenly bodies, of objects found in daily experiences (e.g., balls, birds, cars), and of 
the tiny particles (atoms, molecules, and their component parts) composing all objects and sub- 
stances is the result of interactions of matter and energy. The content statements have been divided 


into topics and subtopics as summarized in exhibit 7. 


> The importance of developing an understanding early and making connections among the Physical, Life, and Earth and Space Sciences has led to 
increased attention on Physical Science in elementary school and prompted consideration of rearranging the usual Earth—life—chemistry—physics 


curriculum sequence. 
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Exhibit 7. Physical Science content topics and subtopics 


Matter 


Properties of matter 
Changes in matter 


Energy 


Forms of energy 
Energy transfer and conservation 


Motion 


Motion at the macroscopic level 


Forces affecting motion 


Matter 


The topic “matter” is divided into two subtopics: properties of matter and changes in matter. 
Conservation of mass, the particulate model of matter, and the periodic table of the elements are the 


concepts that connect these two subtopics and their related principles. 


Properties of Matter 


Matter has physical and chemical properties. Physical properties common to all matter as well 
as those physical properties unique to solids, liquids, and gases are included in the framework, as are 
chemical properties. All objects and substances in the natural world are composed of matter. Mat- 
ter has two fundamental properties: it takes up space and it has inertia; that is, it changes motion 
only when under the influence of a nonzero net force (grade 4). See the following textbox, “Clari- 
fication: A Matter of Mass,’ for more on mass versus weight and the framework’s treatment of this 


distinction. 
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Clarification: A Matter of Mass 


Mass is a property common to all objects. It is the amount of matter (or “stuff”) in an object. The 
more mass an object has, the more inertia (or sluggishness) it displays when attempts are made to 
change its speed or direction. Mass is measured in grams (g) or kilograms (kg) (1 kg = 1,000 g) 
using a beam or electronic balance.* 


Weight, on the other hand, is a measure of the force of attraction (gravitational force) between an 
object and Earth. Every object exerts gravitational force on every other object. The force depends 
on how much mass the objects have and on how far apart they are. Force and weight are measured 


in newtons (N) using a spring scale. 


Changing an object’s position (e.g., from Earth to the Moon) will change its weight, but not its 
mass. For example, on the surface of Earth, a cannonball has a mass of 10 kg and a weight of 98 N. 
On the surface of the Moon, the cannonball still has a mass of 10 kg, but its weight is only 16 N. 
So, the cannonball weighs less on the Moon than on Earth, even though nothing has been taken 
away. Why? Due to the Moon's lesser mass and smaller radius, the force of attraction between the 
Moon and the cannon ball is less than the force of attraction between Earth and the cannonball. 


Hence, it is said that an object on the Moon weighs less than the same object weighs on Earth. 


These concepts of mass and weight are complicated and potentially confusing to 4th grade 
students. Therefore, this framework uses the more familiar term “weight” in grade 4 to stand for 
both weight and mass, and this usage is denoted as “weight (mass).” By grades 8 and 12, students 
are expected to understand the distinction between mass and weight, and both terms will be used 


as appropriate. 


*In current NAEP practice, metric units of measure are used for grades 4, 8, and 12. 


Matter exists in several different physical states, each of which has unique properties (grade 4). 
Three of the most commonly encountered states are solids, liquids, and gases. Shape and compress- 


ibility are examples of properties that distinguish solids, liquids, and gases (grade 4). 


The particulate model of matter can be used to explain and predict the properties of states of 
matter, such as why ice is harder than liquid water and why ice (once formed) has a shape indepen- 
dent of its container while liquid water takes the shape of whatever container it is in (grade 4). In 
the particulate model of matter, the molecules or atoms of which matter is composed are assumed to 
be tiny particles in motion (grade 8). The motion is translational, rotational, and vibrational (grade 


12). This model can be used to explain the properties of solids, liquids, and gases as well as changes 


27 | 


of state. The particulate model can be used to explain the unique properties of water, as described in 


the following textbox. 


Clarification: Unique Properties of Water 


Grade 12: Matter—Changes in Matter 
P12.5: Changes of state require a transfer of energy. Water has a very high specific heat, meaning 


it can absorb a large amount of energy while producing only small changes in temperature. 


The unique properties of water have important consequences for Earth systems and Life Science, 
including the origin and existence of life on Earth. Understanding the substance of water requires 


knowledge across the Physical Science categories of Matter, Energy, and Motion. 


As with all kinds of matter, water’s unique properties can be explained by the shape of its mole- 
cule, the forces between its molecules in solid (ice) and liquid states, and the resulting arrange- 
ment of molecules in solid and liquid states. In particular, in the solid state, the molecular bonds 
are such that they separate the molecules more than in the liquid state, resulting in ice being less 
dense than liquid water. The strong intermolecular forces of liquid water account for its high spe- 
cific heat. (The specific heat of a substance is the amount of energy required to change 1 g of the 
substance by 1 degree centigrade.) 


The detailed structures of molecules and the atoms that compose them serve as models that 
explain the forces of attraction between molecules. The structure of atoms, especially the outermost 
electrons, explains the chemical properties of the elements and the formation of the chemical bonds 
made and broken during chemical reactions (grade 12). The Periodic Table of the Elements (intro- 
duced at grade 8) is another way in which order can be made out of the complexity of the variety 
of types of matter. (In grade 8, the emphasis is on observed periodicity of properties.) The Periodic 
Table demonstrates the relationship between the atomic number of the elements and their chemical 
and physical properties and provides a structure for inquiry into the characteristics of the chemical 
elements (grade 12). 


Two classes of chemical substances serve as exemplars of chemical properties: metals (elements) 
and acids (compounds). A chemical property of metals is to react with nonmetals to form salts. One 
of the properties of acids is the formation of characteristic colors when interacting with acid/base 


indicators and the interaction with bases to produce salts and water (i.e., neutralization) (grade 8). 
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Changes in Matter 


Matter can undergo a variety of changes. Changes are physical if the relationships between 
the molecules of the material are changed, such as from a solid to a liquid or from a liquid to a gas 
(grade 4). When matter undergoes a physical change, generally no changes occur in the structure of 
the molecules or atoms composing the matter (grade 8), although there are exceptions (e.g., sulfur). 
Changes are chemical if they involve the rearrangement of how atoms are bound to one another, 
thereby changing the molecules of the material. These are changes in the configuration of the outer- 
most electron shell surrounding the nuclei of the interacting atoms (grade 12). Changes are nuclear 
if the particles are emitted from or absorbed into the nucleus of the atom, thereby changing the 


atoms themselves into isotopes or different elements (grade 12). 


The fact that mass is conserved when matter undergoes physical and chemical changes is a 
powerful principle for understanding the natural world and was influential in the development of 
chemical theory. Adherence to the principle discourages the conclusion that something “disappears” 


(as water seems to disappear from a puddle) and encourages the search for the “missing” matter. 


Most nuclear reactions, involving changes in the nuclei of atoms, are very high energy and result 
in the formation of elements or nuclei different from those that began the process. In nuclear reac- 


tions, a measurable amount of mass is converted into energy (grade 12). 


Energy 


The topic “energy” is divided into two subtopics; one addresses the forms of energy and the 


other addresses energy transfer and conservation.‘ 


Forms of Energy 


Knowing the characteristics of familiar forms of energy (grade 4) and the scientific categories 
of potential and kinetic energy (grade 8) can lead to an understanding that, for the most part, the 
natural world can be explained and is predictable. The most basic characteristics of thermal, light, 
sound, electrical, and mechanical energy and the relationship between changes in the natural world 
and energy are included in the framework.’ For example, the fact that two objects, one at a higher 
temperature than the other, come to the same temperature when placed in contact with each other is 


a familiar experience. Heat as a concept can be used to explain this experience (grade 8). 


4 There are different approaches to helping students understand the concept of energy and related observable phenomena, such as light, heat, and 
sound. These differences are reflected in the framework’s reference documents and have therefore influenced the framing of the Physical Science, Life 
Science, and Earth and Space Sciences content statements. 

> The term “heat” in Physical Science is used to stand for thermal energy in grade 4; this usage is denoted as “heat (thermal energy).’ “Thermal energy” 
is used in grades 8 and 12. 
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Energy Transfer and Conservation 


The fact that energy is conserved can be demonstrated by keeping track of the familiar forms of 
energy as they are transferred from one object to another. The chemical potential energy in a bat- 
tery is transferred by electric current to a light bulb, which in turn transfers the energy in the form 
of heat (thermal energy) and light to its surroundings (grade 4). The energy stored in the battery 
decreases as its surroundings are heated. The loss in chemical potential energy equals the light and 
heat (thermal energy) transferred by the bulb and the wires to their surroundings. Quantitative 
accounting is complex; however, on a qualitative basis, both the ability to trace energy transfer and 
the understanding that energy is conserved (grade 8) are of great explanatory and predictive value. 
Chemical reactions either release energy to the surroundings or cause energy to flow from the sur- 
roundings into the system (grade 12). The Sun as the main energy source for the Earth provides an 
opportunity at all grade levels to make important connections between the science disciplines (see 


the following textbox). 


Crosscutting Content: Energy Sources and Transfer 


When a sufficiently high temperature occurs in the Sun because of gravitational attraction, 
nuclear reactions take place. These reactions result in the transfer of energy from nuclei to their 
surroundings. At the same time, those high temperatures cause the Sun to radiate visible light and 
many other forms of electromagnetic waves. A small fraction of this light energy reaches Earth, 
heating the land, air, and water. Some of this energy causes some water to evaporate. The water vapor 
travels high into the atmosphere, thus increasing its gravitational potential energy. There it cools 
and condenses, some of it falling as rain into reservoirs. At many dams, some of this water is directed 
downhill through tubes, resulting in the transfer of gravitational potential energy to the descend- 
ing water as kinetic energy. This water is then used to turn turbines, and energy is transferred from 
the moving water to electrical appliances through circuits and power lines. Accordingly, the energy 
used to power commonplace objects such as a light bulb, TV, radio, or stereo can be traced back to 


nuclear reactions deep inside the Sun. 


Consider also the transfer of energy that occurs as a diver falls through air into water. When 
the diver is initially poised on a cliff high above a lake’s surface, one says that the diver has potential 
energy with respect to the air and water below. As the diver falls, her speed (kinetic energy) increases 
as her potential energy decreases. Her body transfers energy to the medium through which she falls; 
that is, the diver’s body rubs against the air and water (heating them) and exerts force on the air and 
water (moving them aside). When the diver finally comes to rest in the lake, some or most of her 


potential energy has been transferred to heating and setting into motion the air and water through 


which she fell. 
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The following grade 12 content statements illustrate the crosscutting nature of energy sources 


and transfer. They are not intended to represent an exhaustive catalog of all statements related to this 


crosscutting content. 


Physical Science 


Earth and Space Sciences 


P12.11: Fission and fusion are reactions involv- 
ing changes in the nuclei of atoms. Fission is the 
splitting of a large nucleus into smaller nuclei and 
particles. Fusion involves joining of two relatively 


light nuclei at extremely high temperature and 


pressure. Fusion is the process responsible for the 


E12.9: Earth systems have internal and external 
sources of energy, both of which create heat. The 
Sun is the major external source of energy. Two 


primary sources of internal energy are the decay of 
radioactive isotopes and the gravitational energy 
from Earth’s original formation. 


energy of the Sun and other stars. (Underlining is 
used here and on pp. 43 and 56 to link segments of 


the content statements across content areas.) 


Motion 


The topic “motion” is divided into two subtopics; one addresses motion at the macroscopic level 


and the other addresses the forces that affect motion. 


Motion at the Macroscopic Level 


Objects observed in daily life undergo different kinds of motion (grade 4). The framework 
distinguishes three kinds of motion (translational, rotational, and vibrational) and emphasizes the 
translational motion of objects in the natural environment (grade 12). Translational motion is more 
difficult to describe than it appears because descriptions depend on the position of the observer and 
the frame of reference used. Speed (grades 4 and 8), velocity (grade 12), and acceleration (grade 12) 
of objects in translational motion are described in terms of change in direction and position in a 


time interval. 


Forces Affecting Motion 


It takes energy to change the motion of objects. The energy change is understood in terms of 
forces. For example, it takes energy for a baseball pitcher to set the ball in motion toward the batter. 
Also, pushes and pulls applied to objects often result in changes in motion (grade 4). Principles ger- 
mane to the relationship of forces and motion serve to motivate the search for forces when objects 
change their motion or when an object remains at rest even though it seems that the forces acting on 


it should result in setting it in motion (grade 8). 
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Some forces act through physical contact of objects while others act at a distance. The force 
of a bat on a ball and the downward push of a lead block resting on a tabletop are contact forces. 
Gravitational and magnetic forces act at a distance (grade 8). Magnets do not need to be in contact 
to attract or repel each other. The Earth and an airplane do not need to be in contact for a force of 
attraction to exist between them. Qualitative relationships (grade 8) and quantitative relationships 
(grade 12) between the mass of an object, the magnitude and direction of the net force on the 


object, and its acceleration are powerful ideas to explain and predict changes in the natural world. 
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Exhibit 8. Physical Science content statements for grades 4, 8, and 12 


Grade 4 


Grade 8 


Grade 12 


Properties of Matter: Physical properties common to all objects and substances and physical properties 
common to solids, liquids, and gases (4); chemical properties, particulate nature of matter, and the Peri- 
odic Table of the Elements (8); characteristics of subatomic particles and atomic structure (12). 


P4.1: Objects and substances 
have properties. Weight (mass) 
and volume are properties that 
can be measured using appropri- 
ate tools.* 


P4.2: Objects vary in the extent 
to which they absorb and reflect 
light and conduct heat (thermal 
energy) and electricity. 


P4.3: Matter exists in several dif- 
ferent states; the most common 
states are solid, liquid, and gas. 
Each state of matter has unique 
properties. For instance, gases are 
easily compressed while solids 
and liquids are not. The shape 

of a solid is independent of its 
container; liquids and gases take 
the shape of their containers. 


P4.4: Some objects are composed 
of a single substance; others are 
composed of more than one 
substance. 


P4.5: Magnets can repel or 
attract other magnets. They can 
also attract certain nonmagnetic 
objects at a distance. 


P8.1: Properties of solids, liquids, 
and gases are explained by a 
model of matter that is composed 
of tiny particles in motion. 


P8.2: Chemical properties of 
substances are explained by 
the arrangement of atoms and 
molecules. 


P8.3: All substances are 
composed of one or more of 
approximately 100 elements. 

The periodic table organizes the 
elements into families of elements 
with similar properties. 

P8.4: Elements are a class of sub- 
stances composed of a single kind 
of atom. Compounds are com- 
posed of two or more different 
elements. Each element and com- 
pound has physical and chemical 
properties, such as boiling point, 
density, color, and conductivity, 
which are independent of the 
amount of the sample.t 


P8.5: Substances are classified 
according to their physical and 
chemical properties. Metals and 
acids are examples of such classes. 
Metals are a class of elements 

that exhibit common physical 
properties such as conductivity 
and common chemical properties 
such as reacting with nonmet- 

als to produce salts. Acids are a 
class of compounds that exhibit 
common chemical properties, 
including a sour taste, characteris- 
tic color changes with litmus and 
other acid/base indicators, and 
the tendency to react with bases 
to produce a salt and water. 


P12.1: Differences in the phys- 
ical properties of solids, liquids, 
and gases are explained by the 
ways in which the atoms, ions, or 
molecules of the substances are 
arranged and the strength of the 
forces of attraction between the 
atoms, ions, or molecules. 


P12.2: Electrons, protons, and 
neutrons are parts of the atom 
and have measurable properties, 
including mass and, in the case 
of protons and electrons, charge. 
The nuclei of atoms are com- 
posed of protons and neutrons. A 
kind of force that is only evident 
at nuclear distances holds the 
particles of the nucleus together 
against the electrical repulsion 
between the protons. 


P12.3: In the Periodic Table, ele- 
ments are arranged according to 
the number of protons (called the 
atomic number). This organiza- 
tion illustrates commonality and 
patterns of physical and chemical 
properties among the elements. 


P12.4: In a neutral atom, the 
positively charged nucleus is 
surrounded by the same number 
of negatively charged electrons. 
Atoms of an element whose 
nuclei have different numbers of 
neutrons are called isotopes. 


ee textbox on p. or more information on the distinction between wei and mass. 
* See textb p. 27 fe format the distinction bet ght and 


t+ Although this content statement generally holds true, some compounds decompose before boiling. 
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Exhibit 8 (continued). Physical Science content statements for grades 4, 8, and 12 


Grade 4 


Grade 8 


molecular level during chemical changes (12). 


Grade 12 


Changes in Matter: Changes of state (4); physical and chemical changes and conservation of mass (8); 


particulate nature of matter, unique physical characteristics of water, and changes at the atomic and 


P4.6: One way to change matter 
from one state to another and 
back again is by heating and 


cooling. 


P8.6: Changes of state are 
explained by a model of matter 
composed of tiny particles that 
are in motion. When substances 
undergo changes of state, neither 
atoms nor molecules themselves 
are changed in structure. Mass 

is conserved when substances 
undergo changes of state. 


P8.7: Chemical changes can 
occur when two substances, ele- 
ments, or compounds react and 
produce one or more different 
substances whose physical and 
chemical properties are different 
from the reacting substances. 
When substances undergo chemi- 
cal change, the number and kinds 
of atoms in the reactants are the 
same as the number and kinds 

of atoms in the products. Mass 

is conserved when substances 
undergo chemical change. The 
mass of the reactants is the same 
as the mass of the products. 


P12.5: Changes of state require 
a transfer of energy. Water has a 
very high specific heat, meaning 
it can absorb a large amount of 
energy while producing only 
small changes in temperature. 


P12.6: An atom’s electron 
configuration, particularly of the 
outermost electrons, determines 
how the atom can interact with 
other atoms. The interactions 
between atoms that hold them 
together in molecules or between 
oppositely charged ions are called 
chemical bonds. 


P12.7: A large number of 
important reactions involve 

the transfer of either electrons 
(oxidation/reduction reactions) 
or hydrogen ions (acid/base 
reactions) between reacting 

ions, molecules, or atoms. In 
other chemical reactions, atoms 
interact with one another by 
sharing electrons to create a 
bond. An important example is 
carbon atoms, which can bond 
to one another in chains, rings, 
and branching networks to form, 
along with other kinds of atoms 
(hydrogen, oxygen, nitrogen, and 
sulfur), a variety of structures, 
including synthetic polymers, 
oils, and the large molecules 
essential to life. 


+ See textbox on p. 28 for more information on the unique properties of water. 
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Exhibit 8 (continued). Physical Science content statements for grades 4, 8, and 12 


Grade 4 


Grade 8 


the Sun (8); nuclear energy and waves (12). 


Grade 12 


Forms of Energy: Examples of forms of energy (4); kinetic energy, potential energy, and light energy from 


P4.7: Heat (thermal energy), 
electricity, light, and sound are 
forms of energy.§ 


P4.8: Heat (thermal energy) 
results when substances burn, 
when certain kinds of materials 
rub against each other, and when 
electricity flows though wires. 
Metals are good conductors of 
heat (thermal energy) and elec- 
tricity. Increasing the temperature 
of any substance requires the 
addition of energy. 


P4.9: Light travels in straight 
lines. When light strikes sub- 
stances and objects through 
which it cannot pass, shad- 
ows result. When light travels 
obliquely from one substance 
to another (air and water), it 
changes direction. 


P4.10: Vibrating objects produce 
sound. The pitch of sound can 

be varied by changing the rate of 
vibration. 


P8.8: Objects and substances in 
motion have kinetic energy. For 
example, a moving baseball can 
break a window; water flowing 
down a stream moves pebbles and 
floating objects along with it. 


P8.9: Three forms of potential 
energy are gravitational, elastic, 
and chemical. Gravitational poten- 
tial energy changes in a system as 
the relative positions of objects are 
changed. Objects can have elastic 
potential energy due to their 
compression, or chemical poten- 
tial energy due to the nature and 
arrangement of the atoms. 


P8.10: Energy is transferred from 
place to place. Light energy from 
the Sun travels through space to 
Earth (radiation). Thermal energy 
travels from a flame through the 
metal of a cooking pan to the 
water in the pan (conduction). 
Air warmed by a fireplace moves 
around a room (convection). 
Waves (including sound and 
seismic waves, waves on water, 
and light waves) have energy and 
transfer energy when they interact 
with matter. 


P8.11: A tiny fraction of the 

light energy from the Sun reaches 
Earth. Light energy from the Sun 
is Earth’s primary source of energy, 
heating Earth surfaces and provid- 
ing the energy that results in wind, 
ocean currents, and storms. 


P12.8: Atoms and molecules that 
compose matter are in constant 
motion (translational, rotational, 
or vibrational). 


P12.9: Energy may be trans- 
ferred from one object to another 
during collisions. 


P 12.10: Electromagnetic waves 
are produced by changing the 
motion of charges or by changing 
magnetic fields. The energy of 
electromagnetic waves is trans- 
ferred to matter in packets. The 
energy content of the packets 

is directly proportional to the 
frequency of the electromagnetic 


waves. 


P12.11: Fission and fusion are 
reactions involving changes in 
the nuclei of atoms. Fission is 
the splitting of a large nucleus 
into smaller nuclei and particles. 
Fusion involves joining two 
relatively light nuclei at extremely 
high temperature and pressure. 
Fusion is the process responsible 
for the energy of the Sun and 
other stars. 


§ See footnote 5 on p. 29 for more information on the framework’s use of the terms “heat” and “thermal 


energy.” 
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Exhibit 8 (continued). Physical Science content statements for grades 4, 8, and 12 


Grade 4 


reactions (12). 


Grade 8 


Grade 12 


Energy Transfer and Conservation: Electrical circuits (4); energy transfer and conservation of energy 
(8); translational, rotational, and vibrational energy of atoms and molecules, and chemical and nuclear 


P4.11: Electricity flowing 
through an electrical circuit 
produces magnetic effects in 
the wires. In an electrical circuit 
containing a battery, a bulb, and 
a bell, energy from the battery is 
transferred to the bulb and the 
bell, which in turn transfer the 
energy to their surroundings as 
light, sound, and heat (thermal 


energy). 


P8.12: When energy is trans- 
ferred from one system to 
another, the quantity of energy 
before transfer equals the quan- 
tity of energy after transfer. For 
example, as an object falls, its 
potential energy decreases as 
its speed, and consequently, its 
kinetic energy increases. While 
an object is falling, some of the 
object’s kinetic energy is trans- 
ferred to the medium through 
which it falls, setting the medium 
into motion and heating it. 


P8.13: Nuclear reactions take 
place in the Sun. In plants, light 
from the Sun is transferred to 
oxygen and carbon compounds, 
which, in combination, have 
chemical potential energy 


(photosynthesis). 


P 12.12: Heating increases the 
translational, rotational, and 
vibrational energy of the atoms 
composing elements and the 
molecules or ions composing 
compounds. As the translational 
energy of the atoms, molecules, 
or ions increases, the temperature 
of the matter increases. Heating 
a sample of a crystalline solid 
increases the vibrational energy 
of the atoms, molecules, or ions. 
When the vibrational energy 
becomes great enough, the crys- 
talline structure breaks down and 
the solid melts. 


P 12.13: The potential energy 
of an object on Earth’s surface is 
increased when the object’s posi- 
tion is changed from one closer 
to Earth’s surface to one farther 
from Earth’s surface. 


P 12.14: Chemical reactions 
either release energy to the envi- 
ronment (exothermic) or absorb 
energy from the environment 
(endothermic). 


P 12.15: Nuclear reactions 
(fission and fusion) convert very 
small amounts of matter into 
appreciable amounts of energy. 


P12.16: Total energy is con- 
served in a closed system. 
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Exhibit 8 (continued). Physical Science content statements for grades 4, 8, and 12 


Grade 4 


Grade 8 


Grade 12 


Motion at the Macroscopic Level: Descriptions of position and motion (4); speed as a quantitative 


description of motion and graphical representations of speed (8); velocity and acceleration as quantitative 


descriptions of motion and the representation of linear velocity and acceleration in tables and graphs (12). 


P4.12: An object's position 

can be described by locating the 
object relative to other objects or 
a background. The description 
of an object’s motion from one 
observer’s view may be different 
from that reported from a differ- 
ent observer’s view. 


P4.13: An object is in motion 
when its position is changing. 
The speed of an object is defined 
by how far it travels divided by 
the amount of time it took to 
travel that far. 


P8.14: An object’s motion can 
be described by its speed and the 
direction in which it is moving. 
An object’s position can be mea- 
sured and graphed as a function 
of time. An object’s speed can 

be measured and graphed as a 
function of time. 


P12.17: The motion of an object 
can be described by its position 
and velocity as functions of time 
and by its average speed and aver- 
age acceleration during intervals 
of time. 


P12.18: Objects undergo 
different kinds of motion 
(translational, rotational, and 
vibrational). 
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Exhibit 8 (continued). Physical Science content statements for grades 4, 8, and 12 


Grade 4 


Grade 8 


Grade 12 


Forces Affecting Motion: The association of changes in motion with forces and the association of objects 
falling toward Earth with gravitational force (4); qualitative descriptions of magnitude and direction as 
characteristics of forces, addition of forces, contact forces, forces that act at a distance, and net force on an 
object and its relationship to the object’s motion (8); quantitative descriptions of universal gravitational 
and electric forces, and relationships among force, mass, and acceleration (12). 


P4.14: The motion of objects can 
be changed by pushing or pulling. 
The size of the change is related 
to the size of the force (push or 
pull) and the weight (mass) of 
the object on which the force is 
exerted. When an object does not 
move in response to a push or a 
pull, it is because another push or 
pull (friction) is being applied by 
the environment. 

P4.15: Earth pulls down on all 
objects with a force called gravity. 
With a few exceptions (heli- 
um-filled balloons), objects fall to 
the ground no matter where the 
object is on Earth. 


P8.15: Some forces between 
objects act when the objects are 
in direct contact or when they are 
not touching. Magnetic, electri- 
cal, and gravitational forces can 
act at a distance. 


P8.16: Forces have magnitude 
and direction. Forces can be 
added. The net force on an 

object is the sum of all the forces 
acting on the object. A nonzero 
net force on an object changes 
the object’s motion; that is, the 
object’s speed and/or direction 
of motion changes. A net force of 
zero on an object does not change 
the object’s motion; that is, the 
object remains at rest or continues 
to move at a constant speed ina 
straight line. 


P 12.19: The motion of an object 
changes only when a net force is 
applied. 

P 12.20: The magnitude of 
acceleration of an object depends 
directly on the strength of the net 
force and inversely on the mass 

of the object. This relationship 
(a=Fnet/m) is independent of the 
nature of the force. 


P 12.21: Whenever one object 
exerts force on another, a force 
equal in magnitude and opposite 
in direction is exerted by the sec- 
ond object back on the first object. 
In closed systems, momentum is 
the quantity of motion that is con- 
served. Conservation of momen- 
tum can be used to help validate 
the relationship a=Fnet/m. 


P 12.22: Gravitation is a universal 
attractive force that each mass 
exerts on any other mass. The 
strength of the gravitational force 
between two masses is propor- 
tional to the masses and inversely 
proportional to the square of the 
distance between them. 


P 12.23: Electric force is a univer- 
sal force that exists between any 
two charged objects. Opposite 
charges attract while like charges 
repel. The strength of the electric 
force is proportional to the magni- 
tudes of the charges and inversely 
proportional to the square of the 
distance between them. Between 
any two charged particles, the 
electric force is vastly greater than 
the gravitational force. 
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Life Science 


Life Science principles are essential for understanding the functioning of living organisms and 
their interactions with their environment. In addition, Life Science principles are crucial for under- 
standing advances in science and technology and appreciating their implications for social and 
personal decisions. For example, the following discoveries of the past 25 years rely on understand- 
ing basic ideas in life science: publication of the human genome and genomes of other organisms, 
ability to monitor the oxygen level of specific regions of the brain, and depletion of the ozone layer 
by human activities. The media regularly ask questions related to health and disease, such as what 
constitutes a healthy lifestyle and how to deal with the mutability of bacteria, viruses, and parasites 
that thwart efforts to develop antibiotics and vaccines. Although science does not currently provide 
complete answers to these types of questions, it provides the tools for understanding and addressing 
them. 


Understanding principles in Life Science is inextricably linked with understanding principles in 
Physical Science and Earth and Space Sciences. “Living organisms are made of the same components 


as all other matter, involve the same kind of transformations of energy, and move using the same 


basic kinds of forces” (AAAS 1994, p. 59). 


Understanding living systems and their interactions with their environment requires not only an 
understanding of various levels of biological organization—molecules, cells, tissues/organs, organ- 
isms, populations, ecosystems—but also an understanding of interactions (including the transfer 
of information) within and across these levels and how they can change over time. For example, 
understanding how populations of organisms change over time is greatly facilitated by understand- 
ing the changes that occur in DNA molecules. These changes are manifest in an organism’s traits and 
may affect its ability to survive and reproduce, which can lead to changing proportions of traits in 
populations over time. 


As summarized in exhibit 9, the Life Science content statements are sorted into topics and sub- 
topics that, collectively, address structure, function, and patterns of change in living systems. How- 
ever, any attempt to organize Life Science by a linear set of topics and subtopics, such as those listed 


below, is somewhat arbitrary. The overlap is evident in exhibit 10, Life Science Content Statements 


for Grades 4, 8, and 12. 
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Exhibit 9. Life Science content topics and subtopics 


Structures and Functions of Living Systems 


Organization and development 
Matter and energy transformations 
Interdependence 


Changes in Living Systems 


Heredity and reproduction 
Evolution and diversity 


Structures and Functions of Living Systems 


This topic comprises the ways in which living systems are organized and how living systems carry 
out their life functions. Reasoning about living systems often involves relating different levels of 
organization, from the molecule to the biosphere, and understanding how living systems are struc- 
tured at each level. The functions of living systems at these levels, particularly how they transform 
matter and energy, are included, as are the interactions among living systems and how they depend 


on one another to carry out their functions. 


Organization and Development 


As it was pointed out early in the 20th century, “Long ago it became evident that the key to 
every biological problem must finally be sought in the cell; for every living organism is, or at some 
time has been, a cell” (Wilson 1928, p. 1). All living things are made up of cells whose work is 
carried out by many different types of molecules. Cellular and molecular biology has the power to 
explain a wide variety of phenomena related to the organization and development of living systems, 
such as synthesis and reproduction, the extraction of energy from food, and regulation. Living 
organisms have a variety of observable features that enable them to obtain food and reproduce 
(grade 4). The functions of living organisms are carried out at different levels of organization. In 
multi-cellular organisms, cells form organs and organ systems (grade 8). Organisms are subsystems 
of populations, communities, ecosystems, and the biosphere. Cellular processes are carried out by 
molecules, particularly proteins. These processes are regulated, both internally and externally, by 
the environments in which cells exist, including local environments that lead to cell differentiation 


during the development of multicellular organisms (grade 12). 
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Matter and Energy Transformations 


Matter and energy transformations are involved in all life processes, such as photosynthesis, 


growth and repair, cellular respiration, and the need of living systems for continual input of energy. 


All single-celled and multicellular organisms have the same basic needs: water, air, a source of 
energy and materials for growth and repair, waste disposal, and conditions for growth and reproduc- 
tion (grade 4). In terms of matter and energy transformations, the source of food is the distinguish- 


ing difference between plants and animals (see textbox below). 


Clarification: Food 


Both plants and animals require a source of energy and materials for growth and repair, and both 
plants and animals use high-energy compounds as a source of fuel and building material. Plants 

are distinguished from animals by the fact that plants have the capability (through photosynthe- 
sis) to take energy from light to form higher energy molecules containing carbon, hydrogen, and 


oxygen (carbohydrates) from lower energy molecules. 


Plants are similar to animals in that, to make other molecules for their growth and reproduction, 
they use the energy that is released as carbohydrates react with oxygen. In making these other 
molecules, plants use the breakdown products of carbohydrates, along with minerals from the soil 
and from fertilizers (known colloquially as “plant foods”), as building blocks. Plants also synthe- 
size substances (carbohydrates, fats, proteins, vitamins) that are components of foods eaten by 


animals. 


Therefore, although synthesis and breakdown are common to both plants and animals, photo- 


synthesis (the conversion of light energy into stored chemical energy) is unique to plants, making 


them the primary source of energy for all animals. 


Basic needs are connected with the processes of growth and metabolism. Organisms are made 
up of carbon-containing molecules; these molecules originate in molecules that plants assemble 
from carbon dioxide and water. In converting carbon-containing molecules back to water and car- 
bon dioxide, organisms release energy, making some of it available to support life functions (grade 
8). Matter and energy transformations in cells, organisms, and ecosystems have a chemical basis 
(grade 12). The following textbox on the flow of energy through an ecosystem illustrates principles 
that cut across the content areas. 
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Crosscutting Content: Uses, Transformations, and Conservation of Energy 


The principles of energy uses, transformations, and conservation hold true across different types 
of systems. These systems include biological organisms, Earth systems, ecosystems (combining 
both life forms and their physical environment), the solar system, other systems in the universe, 
and human-designed systems. 


However complex the workings of living organisms, they share with all other natural sys- 
tems the same physical principles of the conservation and transformation of matter and 
energy. Over long spans of time, matter and energy are transformed among living things, 
and between them and the physical environment. In these grand-scale cycles, the total 
amount of matter and energy remains constant, even though their form and location 


undergo continual change. 


Almost all life on earth is ultimately maintained by transformations of energy from the 
Sun. Plants capture the Sun’s energy and use it to synthesize complex, energy-rich mol- 
ecules (chiefly sugars) from molecules of carbon dioxide and water. These synthesized 
molecules then serve, directly or indirectly, as the source of energy for the plants them- 
selves and ultimately for all animals and decomposer organisms (such as bacteria and 
fungi). This is the food web: The organisms that consume the plants derive energy and 
materials from breaking down the plant molecules, use them to synthesize their own 
structures, and then are themselves consumed by other organisms. At each stage in the 
food web, some energy is stored in newly synthesized structures and some is dissipated 
into the environment as heat produced by the energy-releasing chemical processes in cells. 
A similar energy cycle begins in the oceans with the capture of the Sun’s energy by tiny, 
plant-like organisms. Each successive stage in a food web captures only a small fraction of 
the energy content of organisms it feeds on. 


—-Science for All Americans (AAAS 1994, p. 66). 


The flow of energy in an ecosystem (such as that described above) can be compared to the flow of 
energy illustrated earlier (see “Crosscutting Content: Energy Sources and Transfer” textbox on 

p. 31). They are both identical (the principle) and different (the context). In each case, energy is 
transformed from one form to another; while some is no longer available for human use, it is not 
lost to the system. 


The following grade 4 content statements illustrate the crosscutting nature of uses, transforma- 
tions, and conservation of energy. They are not intended to represent an exhaustive catalog of all 


statements related to this crosscutting content. 


| 42 


Physical Science 


Life Science 


Earth and Space Sciences 


P4.7: Heat (thermal energy), 
electricity, light, and sound are 


L4.2: Organisms have basic 
needs. Animals require air, 


E4.7: The Sun warms the land, 
air, and water and helps plants 


forms of energy. water, and a source of energy row. 
and building material for 
growth and repair. Plants also 
require light. 
Interdependence 


The interaction of species in an ecosystem, the dynamics of population growth and decline, the 
use of resources by multiple species, the impact of species on their environment, and the complex 
interactions among all of these forces have enormous consequences for the survival of all species, 
including humans. 


All animals and most plants depend on both other organisms and their environments for their 
basic needs (grade 4). Organisms interact with one another in a variety of ways, such as producer/ 
consumer, predator/prey, and parasite/host. In addition to competition among organisms, the size 
of populations depends on environmental conditions such as the availability of water, light, and 
other suitable conditions (grade 8). Ecosystems are characterized by both stability and change, on 


which human populations can have an impact (grade 12). 


Changes in Living Systems 


This topic comprises how organisms reproduce, how they pass genetic information to their off- 
spring, and how genetic information can change as it passes from one generation to the next. Over 
time, these changes can affect the size, diversity, and genetic composition of populations (ie., the 


process of biological evolution). 


Heredity and Reproduction 


Organisms closely resemble their parents; their slight variations can accumulate over many 
generations and result in more obvious differences between organisms and their ancestors. Recent 
advances in biochemistry and cell biology have increased understanding of the mechanisms of inher- 
itance and have enabled the detection of disease-related genes. Such knowledge is making it possible 
to design and produce large quantities of substances to treat disease and, in years to come, may lead 


to cures. 
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All plants and animals (and one-celled organisms) develop and have the capacity to reproduce 
(grade 4). Reproduction, whether sexual or asexual, is a requirement for the survival of a species. 
Characteristics of organisms are influenced by heredity and environment (grade 8). Genetic dif- 
ferences among individuals and species are fundamentally chemical. Different organisms are made 
up of somewhat different proteins. Reproduction involves passing the DNA with instructions for 


making these proteins from one generation to the next with occasional modifications (grade 12). 


Evolution and Diversity 


Earth’s present-day life forms have evolved from common ancestors reaching back to the sim- 
plest one-celled organisms almost 4 billion years ago. Modern ideas about evolution provide a sci- 
entific explanation for three main sets of observable facts about life on Earth: the enormous number 
of different life forms that exist, the systematic similarities in anatomy and molecular chemistry 
seen within that diversity, and the sequences of changes in fossils found in successive layers of rock 
that have been formed over more than a billion years. The modern concept of evolution, including 
natural selection and common descent, provides a unifying principle for understanding the history 
of life on Earth, relationships among all living things, and the dependence of life on the physical 
environment. The concept is so well established that it provides a framework for organizing most of 


biological knowledge into a coherent picture. 


All organisms are similar to and different from other organisms, and some kinds of organisms 
and individuals have advantages in particular environments (grade 4). Preferential survival means 
that differences among individuals in a population affect their ability to survive and reproduce. 
Classification reflects degrees of relatedness among species (grade 8). Evolution is the consequence 
of natural selection and differential reproduction. Natural selection and common descent provide 
the scientific explanation for the history of life on Earth as depicted in the fossil record and as 
indicated by anatomical and chemical similarities evident within the diversity of existing organisms 


(grade 12). 
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Exhibit 10. Life Science content statements for grades 4, 8, and 12 


Grade 4 


L4.1: Organisms need food, 
water, and air; a way to dispose 
of waste; and an environment in 
which they can live.* 


Structures and Functions of Living Systems 


(8), the chemical basis of living systems (12). 


L8.1: All organisms are com- 
posed of cells, from one cell 
only to many cells. About two- 
thirds of the weight of cells is 
accounted for by water, which 
gives cells many of their proper- 
ties. In multicellular organisms, 
specialized cells perform special- 
ized functions. Organs and organ 
systems are composed of cells 
and function to serve the needs 
of cells for food, air, and waste 
removal. The way in which cells 
function is similar in all living 
organisms. 


L8.2: Following fertilization, cell 
division produces a small cluster 
of cells that then differentiate by 
appearance and function to form 
the basic tissues of an embryo. 


Grade 12 


Organization and Development: Basic needs of organisms (4), levels of organization of living systems 


L12.1: Living systems are made 


of complex molecules (including 
carbohydrates, fats, proteins, and 
nucleic acids) that consist mostly 
of a few elements, especially car- 
bon, hydrogen, oxygen, nitrogen, 
and phosphorous. 


L12.2: Cellular processes are 
carried out by many different 
types of molecules, mostly pro- 
teins. Protein molecules are long, 
usually folded chains made from 
combinations of amino-acid 
molecules. Protein molecules 
assemble fats and carbohydrates 
and carry out other cellular 
functions. The function of each 
protein molecule depends on its 
specific sequence of amino acids 
and the shape of the molecule. 


L12.3: Cellular processes are 
regulated both internally and 
externally by environments 

in which cells exist, including 
local environments that lead to 
cell differentiation during the 
development of multicellular 
organisms. During the develop- 
ment of complex multicellular 
organisms, cell differentiation is 
regulated through the expression 
of different genes. 


* See p. 41 for a textbox on “Food.” 


+ Human organs and organ systems are subsumed under this content statement. See the specifications for 


more information. 
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Exhibit 10 (continued). Life Science content statements for grades 4, 8, and 12 


Grade 4 


living systems (12). 


Grade 8 


Structures and Functions of Living 


Grade 12 


Matter and Energy Transformations: The basic needs of organisms for growth (4), the role of carbon 
compounds in growth and metabolism (8), the chemical basis of matter and energy transformation in 


L4.2: Organisms have basic 
needs. Animals require air, 
water, and a source of energy and 
building material for growth and 
repair. Plants also require light. 


L8.3: Cells carry out the many 
functions needed to sustain life. 
They grow and divide, thereby 
producing more cells. Food is 
used to provide energy for the 
work that cells do and is a source 
of the molecular building blocks 
from which needed materials are 
assembled. 


L8.4: Plants are producers; that 
is, they use the energy from light 
to make sugar molecules from 
the atoms of carbon dioxide and 
water.¢ Plants use these sugars 
along with minerals from the soil 
to form fats, proteins, and carbo- 
hydrates. These products can be 
used immediately, incorporated 
into the plant's cells as the plant 
grows, or stored for later use. 


L8.5: All animals, including 
humans, are consumers that meet 
their energy needs by eating other 
organisms or their products. 
Consumers break down the struc- 
tures of the organisms they eat to 
make the materials they need to 
grow and function. Decomposers, 
including bacteria and fungi, use 
dead organisms or their products 
to meet their energy needs. 


L12.4: Plants have the capability 
(through photosynthesis) to take 
energy from light to form higher 
energy sugar molecules contain- 
ing carbon, hydrogen, and oxygen 
from lower energy molecules. 
These sugar molecules can be used 
to make amino acids and other 
carbon-containing (organic) mol- 
ecules and assembled into larger 
molecules with biological activity 
(including proteins, DNA, carbo- 
hydrates, and fats). 


L12.5: The chemical elements 
that make up the molecules of 
living things pass through food 
webs and are combined and 
recombined in different ways. At 
each link in an ecosystem, some 
energy is stored in newly made 
structures, but much is dissipated 
into the environment as heat. 
Continual input of energy from 


sunlight keeps the process going. 


L12.6: As matter cycles and 
energy flows through different 
levels of organization of living 
systems (cells, organs, organisms, 
communities) and between living 
systems and the physical envi- 
ronment, chemical elements are 
recombined in different ways. 
Each recombination results in 
storage and dissipation of energy 
into the environment as heat. 
Matter and energy are conserved 
in each change. 


+ The statement “they use the energy from light” does not imply that energy is converted into matter or that 


energy is lost. See textbox “Crosscutting Content: Uses, Transformations, and Conservation of Energy,’ p.42. 
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Exhibit 10 (continued). Life Science content statements for grades 4, 8, and 12 


Grade 4 


Structures and Functions of Living Systems 


Interdependence: The interdependence of organisms (4), specific types of interdependence (8), 
consequences of interdependence (12). 


L4.3: Organisms interact and are 
interdependent in various ways, 
including providing food and 
shelter to one another. Organ- 
isms can survive only in envi- 
ronments in which their needs 
are met. Some interactions are 
beneficial; others are detrimen- 
tal to the organism and other 
organisms. 


L4.4: When the environment 
changes, some plants and animals 
survive and reproduce; others die 
or move to new locations. 


Grade 8 


L8.6: Two types of organisms 
may interact with one another 
in several ways: They may be ina 
producer/ consumer, predator/ 
prey, or parasite/ host relation- 
ship. Or, one organism may 
scavenge or decompose another. 
Relationships may be competi- 
tive or mutually beneficial. Some 
species have become so adapted 
to each other that neither could 
survive without the other. 


L8.7: The number of organisms 
and populations an ecosystem 
can support depends on the 
biotic resources available and 
abiotic factors, such as quantity 
of light and water, range of tem- 
peratures, and soil composition. 


L8.8: All organisms cause 
changes in the environment 
where they live. Some of these 
changes are detrimental to the 
organisms or other organisms, 
whereas others are beneficial. 


Grade 12 


L12.7: Although the interrela- 
tionships and interdependence of 
organisms may generate biolog- 
ical communities in ecosystems 
that are stable for hundreds or 
thousands of years, ecosystems 
always change when climate 
changes or when one or more 
new species appear as a result of 
migration or local evolution. The 
impact of the human species has 
major consequences for other 
species. 


47 | 


Exhibit 10 (continued). Life Science content statements for grades 4, 8, and 12 


Grade 4 


Changes in Living Systems 


L4.5: Plants and animals have 
life cycles. Both plants and ani- 
mals begin life and develop into 
adults, reproduce, and eventu- 
ally die. The details of this life 
cycle are different for different 
organisms. 


L4.6: Plants and animals closely 
resemble their parents. 


Grade 8 


L8.9: Reproduction is a char- 
acteristic of all living systems; 
because no individual organism 
lives forever, reproduction is 
essential to the continuation of 
every species. Some organisms 
reproduce asexually. Other 
organisms reproduce sexually. 


L8.10: The characteristics of 
organisms are influenced by 
heredity and environment. For 
some characteristics, inheritance 
is more important; for other 
characteristics, interactions 
with the environment are more 
important. 


Grade 12 


Heredity and Reproduction: Life cycles (4), reproduction and the influence of heredity and the environ- 
ment on an offspring’s characteristics (8), the molecular basis of heredity (12). 


L12.8: Hereditary information 
is contained in genes, which are 
located in the chromosomes of 
each cell. A human cell contains 
many thousands of different 
genes. One or many genes can 
determine an inherited trait of an 
individual, and a single gene can 
influence more than one trait. 


L12.9: The genetic information 
encoded in DNA molecules pro- 
vides instructions for assembling 
protein molecules. Genes are seg- 
ments of DNA molecules. Insert- 
ing, deleting, or substituting 
DNA segments can alter genes. 
An altered gene may be passed 
on to every cell that develops 
from it. The resulting features 
may help, harm, or have little 

or no effect on the offspring’s 
success in its environment. 


L12.10: Sorting and recombina- 
tion of genes in sexual reproduc- 
tion results in a great variety of 
possible gene combinations from 
the offspring of any two parents. 
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Exhibit 10 (continued). Life Science content statements for grades 4, 8, and 12 


Grade 4 
Changes in Living Systems 


Grade 8 


Grade 12 


Evolution and Diversity: Differences and adaptations of organisms (4), preferential survival and related- 


L4.7: Different kinds of organ- 
isms have characteristics that 
enable them to survive in differ- 
ent environments. Individuals 
of the same kind differ in their 
characteristics, and sometimes 
the differences give individuals 
an advantage in surviving and 
reproducing. 


ness of organisms (8), the mechanisms of evolutionary change and the history of life on Earth (12). 


L8.11: Individual organisms 
with certain traits in particular 
environments are more likely 
than others to survive and have 
offspring. When an environment 
changes, the advantage or dis- 
advantage of characteristics can 
change. Extinction of a species 
occurs when the environment 
changes and the characteristics 
of a species are insufficient to 
allow survival. Fossils indicate 
that many organisms that lived 
long ago are extinct. Extinction 
of a species is common; most of 
the species that have lived on the 
Earth no longer exist. 


L8.12: Similarities among 
organisms are found in anatom- 
ical features, which can be used 
to infer the degree of relatedness 
among organisms. In classifying 
organisms, biologists consider 
details of internal and external 
structures to be more import- 
ant than behavior or general 
appearance. 


L12.11: Modern ideas about 
evolution (including natural 
selection and common descent) 
provide a scientific explanation 
for the history of life on Earth 
as depicted in the fossil record 
and in the similarities evident 
within the diversity of existing 
organisms. 


L12.12: Molecular evidence 
substantiates the anatomical 
evidence for evolution and pro- 
vides additional detail about the 
sequence in which various lines 
of descent branched. 


L12.13: Evolution is the con- 
sequence of the interactions of 
(1) the potential for a species 

to increase its numbers, (2) the 
genetic variability of offspring 
due to mutation and recombina- 
tion of genes, (3) a finite supply 
of the resources required for life, 
and (4) the ensuing selection 
from environmental pressure of 
those organisms better able to 
survive and leave offspring. 
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Earth and Space Sciences 


The past few decades have brought rapid changes in the character of Earth and Space Sciences. 
The study of Earth has shifted from surface geology and mining toward global change and Earth sys- 
tems; research methods have changed from human observations and mapping to remote sensing and 
computer modeling.® This concept of Earth as a complex and dynamic entity of interrelated subsys- 
tems implies that there is no process or phenomenon within the Earth system that occurs in com- 
plete isolation from other elements of the system. There has also been a shift in goals as advances in 
theory have made it possible to more accurately predict changes in weather and climate; to provide 
life-saving warnings of floods, hurricanes, earthquakes, and volcanic eruptions; and to understand 


how human activities influence ecosystem and climate changes across the globe. 


In space science, similar changes have taken place as a result of new technologies. Successful 
probes to Mars, Jupiter, and Saturn have vastly expanded knowledge of the solar neighborhood. 
The discovery of more than 100 planets outside the solar system has raised new questions about 
the origin of life. Furthermore, advances in ground and space-based telescopes capable of observing 
many different parts of the electromagnetic spectrum with unprecedented detail have revolutionized 
understanding of the structure and evolution of the universe itself. In brief, descriptive methods of 
Earth and Space Sciences have given way to theory-based inquiry and problem-solving approaches 
that have far-reaching consequences with regard to understanding the universe and stewardship of 
planet Earth. 


Changes in Earth and Space Science education are beginning to catch up with advances in 
research. The National Standards emphasize a systems approach to studying Earth, especially at 
the high school level. Some of today’s textbooks pay less attention to describing Earth features and 
focus instead on a systems perspective in which Earth is viewed as a physical system of interrelated 
phenomena, processes, and cycles. Some high school curricula have integrated the traditional Earth 
science disciplines of geology, meteorology, and oceanography with aspects of biology, chemistry, 
and physics to introduce students to a more holistic study of Earth. 


The tools available to students for learning about Earth and space have changed as well, 
although not all of these resources are available to all students. Visualization tools, such as geo- 
graphical information system (GIS) software, have made it possible for Earth science students to 
have direct access to the raw data and models used by scientists. Other Web-based programs allow 
students to view and process satellite images of Earth, to direct a camera on board the Space Shuttle, 
and to access professional telescopes around the world to carry out science projects. In other words, 
the core concepts, subject matter, and tools used by students have undergone profound changes in 
recent decades that mirror many of the advances in Earth and Space Sciences. The data and images 


gathered by these tools could be used as source materials for assessment items. For example, students 


6 “Earth” is capitalized, rather than referred to as “the earth,’ in order to recognize it as one of the planets in the solar system. 
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could examine data on sea surface temperatures and upper atmospheric winds, derived from satellite 
observations, to predict the intensity and track of a hurricane. 


To reflect the importance of this content area, NAEP will include questions about Earth and 


Space Sciences at the 4th-, 8th-, and 12th-grade levels. The content statements have been divided 
into topics and subtopics as summarized in exhibit 11. 


Exhibit 11. Earth and Space Sciences content topics and subtopics 


Earth in Space and Time 


Objects in the universe 
History of Earth 


Earth Structures 


Properties of Earth materials 
Tectonics 


Earth Systems 


Energy in Earth systems 
Climate and weather 
Biogeochemical cycles 


Earth in Space and Time 


Earth in space and time is divided into two subtopics: objects in the universe and history of 


Earth. The idea that “the universe is large and ancient, on scales staggering to the human mind” 


(AAAS 1994, p. 40) connects these subtopics. 


One of the earliest discoveries of the scientific age was that Earth is not the center of the uni- 
verse. It is now known that Earth is a planet in space, one of a family of planets and other bodies that 
circle a yellow star in a vast galaxy of other stars. Like countless other worlds that are known to exist, 
Earth has a beginning and a history. That history can be read by carefully and thoughtfully observ- 
ing the world and the universe. 


Objects in the Universe 


Objects in the sky, such as the Sun and the Moon, have patterns of movement. These patterns 
can be observed through changes in shape or placement in the sky based on time of day or season 
(grade 4). By recognizing these patterns, people have developed calendars and clocks and explained 
such phenomena as Moon phases, eclipses, and seasons (grade 8). 
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It was previously thought that Earth was the center of the universe. However, it is now known 
that the Sun is the central and largest body in the solar system, which includes Earth and other plan- 
ets and their moons as well as other objects such as asteroids and comets. Objects in the solar system 


are kept in predictable motion by the force of gravity (grade 8). 


According to the “big bang” theory, the entire contents of the known universe expanded explo- 
sively into existence from a hot, dense state 13.7 billion years ago. Early in the history of the uni- 
verse, stars coalesced out of clouds of hydrogen and helium and clumped together by gravitational 
attraction into billions of galaxies. When heated to a sufficiently high temperature by gravitational 
attraction, stars begin nuclear reactions, which convert matter to energy and fuse light elements into 


heavier ones (grade 12). 


History of Earth 


Theories of planet formation and radioactive dating of meteorites have led to the conclusion 
that the Sun, Earth, and the rest of the solar system formed from a nebular cloud of dust and gas 4.6 
billion years ago. Early Earth was very different from today’s planet. Initially, there was no life and 
no molecular oxygen in the atmosphere. Evidence shows that one-celled organisms (bacteria) were 
the first forms of life on the planet, appearing about 3.5 billion years ago. These bacteria are thought 
to be responsible for adding oxygen to Earth’s atmosphere, making it possible for a wider diversity of 
life forms to evolve (grade 12). 


Earth processes seen today, such as erosion and mountain building, have made possible the 
measurement of geologic time through methods such as observing rock sequences and using fossils 
to correlate the sequences at various locations. Fossils also provide evidence of how life and envi- 
ronmental conditions have changed (grade 8). Early methods of determining geological time, such 
as the use of index fossils and stratigraphic sequences, allowed for the relative dating of geologic 
events. However, absolute dating was impossible until the discovery that certain radioactive isotopes 
in rocks have known decay rates, making it possible to determine how many years ago a given rock 


sample formed (grade 12). 


Earth’s surface changes over time. Some changes are due to slow processes, such as erosion and 
weathering, and others are due to rapid processes such as volcanic eruptions, landslides, and earth- 
quakes (grade 4). Changes caused by violent earthquakes and volcanic eruptions can be observed on 
a human time scale; on the other hand, many geological processes, such as the building of mountain 
chains and shifting of entire continents, take place over hundreds of millions of years. Water, ice, 


waves, and wind sculpt Earth’s surface to produce distinctive landforms (grade 12). 


| 52 


Earth Structures 


Content statements related to Earth structures are divided into two subtopics: properties 
of Earth materials and tectonics. The study of Earth materials has contributed to understanding 
dynamic processes, which are in turn driven by the movement of vast tectonic plates. Conversely, the 
development of tectonic theory has made it possible to locate and extract Earth materials for a wide 


variety of human uses. 


Properties of Earth Materials 


Earth materials that occur in nature include rocks, minerals, soils, water, and the gases of the 
atmosphere. Natural materials have different properties that sustain plant and animal life (grade 
4). Soil consists of weathered rocks and decomposed organic material from dead plants, animals, 
and bacteria. Soils are often found in layers, with each having a different chemical composition and 
texture (grade 8). Some Earth materials have properties either in their present form or after design 


and modification that make them useful in solving human problems and enhancing the quality of 


life (grade 4). 


Rocks and rock formations bear evidence of the conditions and forces that created them, rang- 
ing from the violent conditions of volcanic eruptions to the slow deposition of sediments. The atmo- 
sphere is a mixture of nitrogen, oxygen, and trace gases that include water vapor. The atmosphere has 


a different physical and chemical composition at different elevations (grade 8). 


Tectonics 


A basic understanding of geological history (described above) forms the foundation for later 
understanding of tectonics. Earth’s internal structure is layered with a lithosphere, hot convecting 
mantle, and dense metallic core. Lithospheric plates, on the scale of continents and oceans, con- 
stantly move at rates of centimeters per year in response to movements in the mantle. Major geolog- 
ical events such as earthquakes, volcanic eruptions, and mountain building result from these plate 


motions (grade 8). 


Although continental drift was first suggested as early as the late 16th century and further 
developed in the early 1900s, it was not widely accepted until more convincing evidence emerged 
as a result of extensive exploration of the sea floor. By the late 1960s, mapping of the Mid-Atlantic 
Ridge, evidence of sea floor spreading, and subduction led to the more general theory of plate tec- 
tonics. The current explanation is that the outward transfer of Earth’s internal heat propels the plates 


comprising Earth’s surface across the face of the globe, pushing the plates apart where magma rises 
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to form mid-ocean ridges, and pulling the edges of plates back down where Earth materials sink into 


the crust at deep trenches (grade 12). 


Earth as a whole has a magnetic field that is detectable at the surface with a compass. Earth’s 
magnetic field is similar to the field of a natural or manmade magnet with north and south poles and 
lines of force. For thousands of years, people have used compasses to aid navigation on land and sea 
(grade 8). Crucial evidence in support of tectonic theory came from studies of the magnetic proper- 


ties of rocks on the ocean floor (grade 12). 


Earth Systems 


Earth systems is divided into three subtopics: energy in Earth systems, climate and weather, 
and biogeochemical cycles. The explorers of the 16th century who circumnavigated the planet were 
the first to become aware of global weather and climate patterns. As science began to mature and 
diversify in the 19th and 20th centuries, those who studied the planet scientifically did so from the 
perspective of the traditional disciplines, such as geology, oceanography, and meteorology. Cur- 
rently, working with vastly improved technologies, most scientists take an Earth systems perspective, 
including the study of how energy moves through Earth systems and the integration of disciplines to 


better understand Earth’s biogeochemical cycles. 


Energy in Earth Systems 


The Sun warms the land, air, and water and helps plants grow (grade 4). The Sun is the major 
source of energy for phenomena on Earth’s surface. It drives convection within the atmosphere and 
oceans, producing winds, ocean currents, and the water cycle. Seasons result from annual variations 
in the intensity of sunlight and length of day due to the tilt of Earth’s rotation axis relative to the 
plane of its yearly orbit around the Sun (grade 8). 


Earth’s systems have internal and external sources of energy, both of which create heat. The Sun 
is the major external source of energy. Two primary sources of internal energy are the decay of radio- 


active isotopes and the gravitational/thermal energy from Earth’s original formation (grade 12). 


Climate and Weather 


Weather changes from day to day and during the seasons. Scientists use tools for recording and 
predicting weather changes (grade 4). Global patterns of atmospheric movement influence local 


weather (grade 8). 
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Climate is determined by energy transfer from the Sun at and near Earth’s surface. This energy 
transfer is influenced by dynamic processes such as cloud cover, atmospheric gases, and Earth’s rota- 
tion, as well as static conditions such as the position of mountain ranges and oceans, seas, and lakes 


(grade 12). Oceans have a major effect on climate because water in the oceans holds a large amount 


of heat (grade 8). 


Biogeochemical Cycles 


Earth is a system containing essentially a fixed amount of each stable chemical atom or element. 
Each element can exist in several different chemical forms. Elements move within and between 
the lithosphere, atmosphere, hydrosphere, and biosphere as part of biogeochemical cycles (see the 
following textbox). Movement of matter through Earth’s systems is driven by Earth’s internal and 
external sources of energy. These movements are often accompanied by a change in the physical and 
chemical properties of the matter. For example, carbon occurs in carbonate rocks such as limestone, 
in coal and other fossil fuels, in the atmosphere as carbon dioxide gas, in water as dissolved carbon 


dioxide, and in all organisms as complex molecules that control the chemistry of life (grade 12). 


Water, which covers the majority of Earth’s surface, circulates through the crust, oceans, and 
atmosphere in what is known as the water cycle. Water evaporates from Earth’s surface, rises and 
cools as it moves to higher elevations, condenses as clouds, falls as rain or snow, and collects in lakes, 


oceans, soil, and underground (grade 8). 


Natural ecosystems provide an array of basic processes that affect humans. These processes 
include maintenance of the quality of the atmosphere, generation of soils, control of the hydrologic 
cycle, disposal of wastes, and recycling of nutrients (grade 12). The supply of many Earth resources 


such as fuels, metals, fresh water, and farmland is limited. 


Humans change environments in ways that can either be beneficial or detrimental for them- 
selves and other organisms (grade 4). Humans have devised methods for extending the use of Earth 
resources through recycling, reuse, and renewal (grade 4). However, other activities (reducing the 
amount of forest cover, increasing the amount and variety of chemicals released into the atmosphere, 
and intensive farming) have changed Earth’s land, oceans, and atmosphere. Studies of plant and 
animal populations have shown that such activities can reduce the number and variety of wild plants 


and animals and sometimes result in the extinction of species (grade 8). 
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Crosscutting Content: Biogeochemical Cycles 


To demonstrate an understanding of biogeochemical cycles, students must draw on their 
knowledge of matter and energy (Physical Science), structures and functions of living systems 
(Life Science), and Earth systems (Earth and Space Sciences). 


Essentially fixed amounts of chemical atoms or elements cycle within the Earth system, and 
energy drives their translocation and transformation. Examples of biogeochemical cycles include 
water, carbon, and nitrogen. The basic processes underlying the translocation of matter (e.g., 
changes of state, gravity) and transformations involving the rearrangement of atoms in chemical 


reactions are described in Physical Science (exhibit 8 on p. 34) and the role of living organisms in 


cycling atoms between inorganic and organic forms is described in Life Science (exhibit 10 on 


p. 46). 


Biogeochemical cycles are described more fully in the Earth Systems section of exhibit 12, 


Earth and Space Science Content Statements for Grades 4, 8, and 12. 


The following grade 12 content statements illustrate the crosscutting nature of biogeochemi- 


cal cycles. They are not intended to represent an exhaustive catalog of all statements related to this 


crosscutting content. 


Physical Science 


Life Science 


Earth and Space Sciences 


P12.7: A large number of 
important reactions involve 

the transfer of either electrons 
(oxidation/reduction reactions) 
or hydrogen ions (acid/base 
reactions) between reacting 
ions, molecules, or atoms. In 
other chemical reactions, atoms 
interact with one another by 
sharing electrons to create a 
bond. An important example 

is carbon atoms, which can 
bond to one another in chains, 
rings, and branching networks 
to form, along with other kinds 


of atoms (hydrogen, oxygen, 


nitrogen, and sulfur) a variety of 


structures, including synthetic 
polymers, oils, and the large 
molecules essential to life. 


L12.5: The chemical elements 
that make up the molecules of 
living things pass through food 
webs and are combined and 
recombined in different ways. 
At each link in an ecosystem, 
some energy is stored in newly 


made structures, but much is 
dissipated into the environment 
as heat. Continual input of 
energy from sunlight keeps the 
process going. 


E12.12: Movement of mat- 

ter through Earth's systems is 
driven by Earth’s internal and 
external sources of energy. These 
movements are often accompa- 
nied by a change in the physical 
and chemical properties of the 
matter. Carbon, for example, 
occurs in carbonate rocks such 
as limestone, in coal and other 
fossil fuels, in the atmosphere as 
carbon dioxide gas, in water as 
dissolved carbon dioxide, and in 


all organisms as complex mole- 
cules that control the chemistry 


of life. 
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Exhibit 12. Earth and Space Sciences content statements for grades 4, 8, and 12 


Grade 4 


Earth in Space and Time 


Objects in the Universe: Patterns in the sky (4), a model of the solar system (8), a vision of the 


universe (12). 


E4.1: Objects in the sky have 
patterns of movement. The Sun, 
for example, appears to move 
across the sky in the same way 
every day, but its path changes 
slowly over the seasons. The 
Moon appears to move across the 
sky on a daily basis much like the 


Sun. 


E4.2: The observable shape of 
the Moon changes from day to 
day in a cycle that lasts about a 
month. 


E8.1: In contrast to an earlier 
theory that Earth is the center 
of the universe, it is now known 
that the Sun, an average star, is 
the central and largest body in 
the solar system. Earth is the 
third planet from the Sun in a 
system that includes seven other 
planets and their moons, as well 
as smaller objects such as aster- 
oids and comets. 


E8.2: Gravity is the force that 
keeps most objects in the solar 
system in regular and predictable 
motion. These motions explain 
such phenomena as the day, the 
year, phases of the Moon, and 
eclipses. 


Grade 12 


E12.1: The origin of the uni- 
verse remains one of the greatest 
questions in science. The “big 
bang” theory places the origin 
approximately 13.7 billion years 
ago when the universe began in 

a hot, dense state. According to 
this theory, the universe has been 
expanding ever since. 


E12.2: Early in the history of 
the universe, matter (primarily 
the light atoms hydrogen and 
helium) clumped together by 
gravitational attraction to form 
countless trillions of stars and 
billions of galaxies. 


E12.3: Stars, like the Sun, 
transform matter into energy in 
nuclear reactions. When hydro- 
gen nuclei fuse to form helium, 
a small amount of matter is 
converted to energy. These and 
other processes in stars have led 
to the formation of all the other 


elements. 
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Exhibit 12 (continued). Earth and Space Sciences 
content statements for grades 4, 8, and 12 


Grade 4 


Earth in Space and Time 


Grade 8 


Grade 12 


History of Earth: Evidence of change (4), estimating the timing and sequence of geologic events (8), 


theories about Earth’s history (12). 


E4.3: The surface of Earth 
changes. Some changes are due 

to slow processes such as ero- 
sion and weathering, and some 
changes are due to rapid processes 
such as landslides, volcanic erup- 
tions, and earthquakes. 
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E8.3: Fossils provide important 
evidence of how life and environ- 
mental conditions have changed 
in a given location. 


E8.4: Earth processes seen today, 
such as erosion and mountain 
building, make it possible to 
measure geologic time through 
methods such as observing rock 
sequences and using fossils to 
correlate the sequences at various 
locations. 


E12.4: Early methods of deter- 
mining geologic time, such as the 
use of index fossils and strati- 
graphic sequences, allowed for 
the relative dating of geological 
events. However, absolute dating 
was impossible until the discovery 
that certain radioactive isotopes 
in rocks have known decay rates, 
making it possible to determine 
how many years ago a given rock 
sample formed. 


E12.5: Theories of planet for- 
mation and radioactive dating 
of meteorites and lunar samples 
have led to the conclusion that 
the Sun, Earth, and the rest of 
the solar system formed from a 
nebular cloud of dust and gas 4.6 


billion years ago. 


E12.6: Early Earth was very 
different from today’s planet. Evi- 
dence for one-celled forms of life 
(bacteria) extends back more than 
3.5 billion years. The evolution 
of life caused dramatic changes in 
the composition of Earth’s atmo- 
sphere, which did not originally 


contain molecular oxygen. 


E12.7: Earth’s current structure 
has been influenced by both 
sporadic and gradual events. 
Changes caused by violent earth- 
quakes and volcanic eruptions 
can be observed on a human time 
scale; however, many geological 
processes, such as the building of 
mountain chains and shifting of 
entire continents, take place over 
hundreds of millions of years. 


Exhibit 12 (continued). Earth and Space Sciences 
content statements for grades 4, 8, and 12 


Grade 4 


atmosphere (8). 


E4.4: Earth materials that occur 
in nature include rocks, minerals, 
soils, water, and the gases of the 
atmosphere. 


E4.5: Natural materials have 
different properties that sustain 
plant and animal life. 


E4.6: Some Earth materials have 
properties either in their present 
form or after design and modi- 
fication that make them useful 
in solving human problems and 
enhancing the quality of life, as 
in the case of materials used for 
building or fuels used for heating 


and transportation. 


Grade 8 


Properties of Earth Materials: Natural and manmade materials (4), soil analysis and layers of the 


E8.5: Rocks and rock formations 
bear evidence of the minerals, 
materials, temperature/pres- 

sure conditions, and forces that 
created them. Some formations 
show evidence that they were 
deposited by volcanic eruptions. 
Others are composed of sand and 
smaller particles that are buried 
and cemented by dissolved min- 
erals to form solid rock again. 
Still others show evidence that 
they were once earlier rock types 
that were exposed to heat and 
pressure until they changed shape 
and, in some cases, melted and 
recrystallized. 


E8.6: Soil consists of weathered 
rocks and decomposed organic 
material from dead plants, ani- 
mals, and bacteria. Soils are often 
found in layers with each having 
a different chemical composition 
and texture. 


E8.7: The atmosphere is a 
mixture of nitrogen, oxygen, and 
trace gases that include water 
vapor. The atmosphere has a 
different physical and chem- 

ical composition at different 
elevations. 
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Exhibit 12 (continued). Earth and Space Sciences 
content statements for grades 4, 8, and 12 


Grade 4 Grade 8 Grade 12 


Tectonics: The basics of tectonic theory and Earth magnetism (8), the physical mechanism that drives 
tectonics and its supporting evidence (12). 


E8.8: Earth is layered with a 
lithosphere; a hot, convecting 
mantle; and a dense, metallic 
core. 


E8.9: Lithospheric plates on the 
scale of continents and oceans 
constantly move at rates of 
centimeters per year in response 
to movements in the mantle. 
Major geological events, such as 
earthquakes, volcanic eruptions, 
and mountain building, result 
from these plate motions. 


E8.10: Earth as a whole has a 
magnetic field that is detectable 
at the surface with a compass. 
Earth’s magnetic field is similar 
to the field of a natural or man- 
made magnet with north and 
south poles and lines of force. 
For thousands of years, people 
have used compasses to aid in 
navigation on land and sea. 


E12.8: Mapping of the Mid-At- 
lantic Ridge, evidence of sea 
floor spreading, and subduction 
provided crucial evidence in 
support of the theory of plate 
tectonics. The theory currently 
explains plate motion as follows: 
the outward transfer of Earth’s 
internal heat propels the plates 
comprising Earth’s surface across 
the face of the globe. Plates are 
pushed apart where magma rises 
to form midocean ridges, and the 
edges of plates are pulled back 
down where Earth materials sink 
into the crust at deep trenches. 
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Exhibit 12 (continued). Earth and Space Sciences 
content statements for grades 4, 8, and 12 


Grade 4 


E4.7: The Sun warms the land, 
air, and water and helps plants 
grow. 


Grade 8 


Energy in Earth Systems: The role of the Sun (4), the Sun’s observable effects (8), internal and external 
sources of energy in Earth systems (12). 


E8.11: The Sun is the major 
source of energy for phenomena 
on Earth’s surface. It provides 
energy for plants to grow and 
drives convection within the 
atmosphere and oceans, produc- 
ing winds, ocean currents, and 
the water cycle. 


E8.12: Seasons result from 
annual variations in the intensity 
of sunlight and length of day, due 
to the tilt of Earth’s rotation axis 
relative to the plane of its yearly 
orbit around the Sun. 


Grade 12 


E12.9: Earth systems have 
internal and external sources 

of energy, both of which create 
heat. The Sun is the major exter- 
nal source of energy. Two pri- 
mary sources of internal energy 
are the decay of radioactive 
isotopes and the gravitational 
energy from Earth’s original 
formation. 


Climate and Weather: Local weather (4), global weather patterns (8), systems that influence climate (12). 


E4.8: Weather changes from day 
to day and during the seasons. 


E4.9: Scientists use tools for 
observing, recording, and pre- 
dicting weather changes from day 
to day and during the seasons. 


E8.13: Global patterns of atmo- 
spheric movement influence local 
weather. Oceans have a major 
effect on climate because water in 
the oceans holds a large amount 


of heat. 


E12.10: Climate is determined 
by energy transfer from the Sun 
at and near Earth’s surface. This 
energy transfer is influenced by 
dynamic processes such as cloud 
cover, atmospheric gases, and 
Earth’s rotation, as well as static 
conditions such as the positions 
of mountain ranges, oceans, seas, 


and lakes. 
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Exhibit 12 (continued). Earth and Space Sciences 
content statements for grades 4, 8, and 12 


Grade 4 


Grade 8 


Grade 12 


Biogeochemical Cycles: Uses of Earth resources (4), natural and human-induced changes in Earth mate- 
rials and systems (8), biogeochemical cycles in Earth systems (12). 


E4.10: The supply of many 
Earth resources such as fuels, 
metals, fresh water, and farmland 
is limited. Humans have devised 
methods for extending the use of 
Earth resources through recy- 
cling, reuse, and renewal. 


E4.11: Humans depend on their 
natural and constructed environ- 
ment. Humans change environ- 
ments in ways that can either 

be beneficial or detrimental for 
themselves and other organisms. 


E8.14: Water, which covers the 
majority of Earth’s surface, circu- 
lates through the crust, oceans, 
and atmosphere in what is 
known as the water cycle. Water 
evaporates from Earth’s surface, 
rises and cools as it moves to 
higher elevations, condenses as 
clouds, falls as rain or snow, and 
collects in lakes, oceans, soil, and 
underground. 


E8.15: Human activities, such 
as reducing the amount of forest 
cover, increasing the amount and 
variety of chemicals released into 
the atmosphere, and intensive 
farming, have changed Earth’s 
land, oceans, and atmosphere. 
Studies of plant and animal pop- 
ulations have shown that such 
activities can reduce the number 
and variety of wild plants and 
animals and sometimes result in 
the extinction of species. 


E12.11: Earth is a system con- 
taining essentially a fixed amount 
of each stable chemical atom 

or element. Most elements can 
exist in several different chemi- 
cal forms. Earth elements move 
within and between the litho- 
sphere, atmosphere, hydrosphere, 
and biosphere as part of biogeo- 
chemical cycles. 


E12.12: Movement of matter 
through Earth’s systems is driven 
by Earth’s internal and external 
sources of energy. These move- 
ments are often accompanied 
by a change in the physical 

and chemical properties of the 
matter. Carbon, for example, 
occurs in carbonate rocks such 
as limestone, in coal and other 
fossil fuels, in the atmosphere as 
carbon dioxide gas, in water as 
dissolved carbon dioxide, and in 
all organisms as complex mole- 
cules that control the chemistry 


of life. 


E12.13: Natural ecosystems 
provide an array of basic pro- 
cesses that affect humans. These 
processes include maintenance 
of the quality of the atmosphere, 
generation of soils, control of 
the hydrologic cycle, disposal 

of wastes, and recycling of 
nutrients. 
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Components of Science Content as Assessment Item Contexts 


Science-literate citizens should be familiar with certain components of science content, such 
as the history and nature of science and the relationship between science and technology. These 
features are highly valued by science educators and are viewed as critical to the teaching and learning 
of science (AAAS 1993; NRC 1996). In chapter three, the nature of science is partially addressed 
through a discussion of scientific inquiry. Similarly, the relationship between science and technol- 
ogy is partially addressed in chapter three through a discussion of technological design. In addition, 
these components of science content will be incorporated into the contexts of assessment items; 
they will not be directly assessed because of time and resource constraints. Chapter four contains 


additional information. 


From Science Content to Science Practices 


This chapter has presented the science content that defines the NAEP Science Assessment 
content domain. The content statements, as presented in this chapter, do not describe students’ 
performances in observable terms. Chapter three contains a description of science practices and 
cognitive demands. It also shows how science content statements can be combined (crossed) with 
science practices to generate performance expectations (ie., descriptions of students’ expected and 
observable performances on the NAEP Science Assessment). Based on these performance expecta- 
tions, assessment items can be developed and then inferences can be derived from student responses 
about what students know and can do in science. Chapter three provides an illustrative example of 


this process. 
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CHAPTER THREE 


SCIENCE PRACTICES 


Chapter two presented content statements that define the key science principles (as well as 
the facts, concepts, laws, and theories) that NAEP will assess. However, NAEP will assess not only 
science content statements but also the ways in which knowledge is used. This chapter defines what 
students should be able to do with the science content statements by articulating key science prac- 
tices that NAEP will assess: Identifying Science Principles, Using Science Principles, Using Scientific 
Inquiry, and Using Technological Design. These practices are useful for generating science-rich 


assessment items. 


To help assessment developers, the science practices can be associated with the cognitive 
demands they place on students. This chapter employs a set of four cognitive demands: (1) “know- 
ing that,’ (2) “knowing how,’ (3) “knowing why,’ and (4) “knowing when and where to apply knowl- 
edge.” These cognitive demands help ensure that NAEP assessment items are developed in a way that 
will elicit the kinds of knowledge and thinking that underlie the framework’s performance expecta- 
tions (as explained later in this chapter). They also provide a tool for interpreting students’ responses 


on the assessment items. 


This chapter shows how science content statements can be combined or crossed with practices 
to generate performance expectations, which then guide the development of assessment items. By 
comparing student responses with the particular science content and practice being assessed, infer- 
ences about what students know (about particular science principles) and can do (with respect to 


particular science practices) are made. 


Two types of textboxes are used throughout this chapter. Clarification textboxes provide details 
on potentially confusing topics, such as the distinction between Identifying Science Principles and 
Using Science Principles. Illustrative item textboxes provide assessment items that exemplify rec- 
ommendations discussed in the text. Answers to selected-response items are indicated within the 
textbox; appendix C contains scoring guides for constructed-response items. Although items in 


these textboxes may assess more than one content statement or practice, only the primary content 
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and practice designations are provided. This follows NAEP practice, which uses only primary desig- 


nations for items in the analysis and reporting of student responses. 


Overview of Practices 


Over the course of human history, people have developed many interconnected and validated 
ideas about the physical and biological world. These ideas have enabled successive generations to 
achieve an increasingly comprehensive and reliable understanding of the natural world. Scientific 
ideas are generated and verified by observing natural phenomena, finding patterns in these obser- 
vations, and constructing theoretical models to explain these patterns.’ The patterns and models 
can in turn be used to describe, measure, classify, explain, and predict other observations. Science 
knowledge is used to reason about the natural world and to improve the quality of scientific thought 
and action. Hence, NAEP will assess how well 4th, 8th, and 12th grade students can engage in the 


following broadly organized science practices: 


e Identifying Science Principles 
e Using Science Principles 

e Using Scientific Inquiry 

e Using Technological Design 


Because these practices are closely related, the categories are not distinct and some overlap is 


expected. 


The ability to communicate accurately and effectively is essential in science, and this expectation 
is a strand that runs across the practices. Accurate and effective communication may include (but is 
not limited to) writing clear instructions that others can follow to carry out an investigation; reading 
and organizing data in tables and graphs; locating information in computer databases; using audio, 
video, multimedia, and other technologies to access, process, and integrate scientific findings; using 
language and scientific terms appropriately; drawing pictures or schematics to aid in descriptions of 
observations; summarizing the results of scientific investigations; and reporting to various audiences 
about facts, explanations, investigations, and data-based alternative explanations and designs (AAAS 
1993; NRC 1996; Partnership for 21st Century Skills 2004). Quantitative reasoning is also funda- 
mental to science. It is the capacity not only to calculate (e.g., determine density given an object’s 
mass and dimensions) but also to model a system (e.g., determine the energy released from a chemi- 


cal reaction). 


7 Because natural phenomena are understood and described based on collected observations, the terms “phenomena” and “observations” are intricately 
intertwined. For ease of communication, the framework uses the term “observations” to represent both specific observations of a natural phenomenon 


and the phenomenon itself. 
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Sources for the Development of Practices 


The framework developers examined a number of sources to develop the short list of practices 
to be assessed in the NAEP Science Assessment. The most important sources were the Science 
as Inquiry sections of the National Standards and chapter 12: Habits of Mind in Benchmarks. 
The committee also consulted the National Standards and Benchmarks sections on Science and 
Technology and The Designed World, and the Validities of Science Inquiry Assessments project 
(Quellmalz et al. 2005). Conducted by SRI International during 2001-05, this project classified 
assessment items according to the inquiry standards discussed in the National Standards. The prac- 
tices described below are found in most of the above sources. Cognitive research on science learning, 


international frameworks, and state standards were also used as reference points. 


Identifying Science Principles 


This category focuses on students’ ability to recognize, recall, define, relate, and represent basic 
science principles specified in the Physical Science, Life Science, and Earth and Space Sciences 
content statements presented in chapter two. The content statements themselves are often closely 
related to one another conceptually. Moreover, the science principles included in the content state- 
ments can be represented in a variety of forms, such as words, pictures, graphs, tables, formulas, and 
diagrams (AAAS 1993; NRC 1996). NAEP will assess students’ ability to describe, measure, or clas- 
sify observations; state or recognize principles included in the content statements; connect closely 
related content statements; and relate different representations of science knowledge. The practices 
assessed in this category draw on declarative knowledge (or “knowing that”), which is described in 
the Cognitive Demands section later in this chapter. Identifying Science Principles comprises the 


following general types of performance expectations: 


e Describe, measure, or classify observations (e.g., describe the position and motion of 
objects; measure temperature; classify relationships between organisms as being predator/ 
prey, parasite/host, producer/consumer). 

e State or recognize correct science principles (e.g., mass is conserved when substances 
undergo changes of state; all organisms are composed of cells; the atmosphere is a mixture 
of nitrogen, oxygen, and trace gases that include water vapor). 

e Demonstrate relationships among closely related science principles (¢.g., connect state- 
ments of Newton’s three laws of motion, relate energy transfer with the water cycle). 

e Demonstrate relationships among different representations of principles (e.g., verbal, sym- 


bolic, diagrammatic) and data patterns (e.g., tables, equations, graphs). 


Identifying Science Principles is integral to all of the other science practices. 


The following two items illustrate the expectation that students will recognize correct science 


principles. The first item assesses students’ ability to correctly identify simple information about the 
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location of bodies within the solar system (declarative knowledge). More than half of the eighth 
graders nationally answered it incorrectly. Thirty-five percent of the students thought that the Moon 
is sometimes closer to the Sun than to Earth. 


Illustrative Item 
The Earth’s Moon is 


A. always much closer to the Sun than it is to the Earth. 

B. always much closer to the Earth than it is to the Sun. 

C. about the same distance from the Sun as it is from the Earth. 

D. sometimes closer to the Sun than it is to the Earth and sometimes closer to the Earth 


than it is to the Sun. 
Key: B 


E8.1, Identifying Science Principles 
Source: NAEP 2000, Grade 8. 


Illustrative Item 


Animals and plants are made up of a number of different chemical elements. What happens to all 


of these elements when animals and plants die? 


A. They die with the animal or plant. 
B. ‘They evaporate into the atmosphere. 
C. They are recycled back into the environment. 


D. They change into different elements. 
Key: C 


L8.5, Identifying Science Principles 
Source: TIMSS 2003, Grade 8.* 


* TIMSS items appearing in the framework are copyrighted © by the International Association 


for the Evaluation of Educational Achievement. 
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Using Science Principles 


Scientific knowledge is useful for making sense of the natural world. Both scientists and 
informed citizens can use patterns in observations and theoretical models to predict and explain 
observations that they make now or that they will make in the future. The practices assessed in this 
category draw primarily on schematic knowledge (or “knowing why”) in addition to declarative 
knowledge, which are described in the Cognitive Demands section later in this chapter. Using Sci- 


ence Principles comprises the following general types of performance expectations: 


e Explain observations of phenomena (using science principles from the content statements). 

e Predict observations of phenomena (using science principles from the content statements, 
including quantitative predictions based on science principles that specify quantitative 
relationships among variables). 

e Suggest examples of observations that illustrate a science principle (e.g., identify examples 
where the net force on an object is zero; provide examples of observations explained by the 
movement of tectonic plates; given partial DNA sequences of organisms, identify likely 
sequences of close relatives). 


e Propose, analyze, and/or evaluate alternative explanations or predictions. 


The following item illustrates the expectation that students will predict phenomena. 


Illustrative Item 


Fg) 


Oak tree Caterpillar 


Look at the food web above. If the corn crop failed one year, what would most likely happen to 
the robin population? Explain your answer. 


(See appendix C for item scoring guides.) 


L8.6, Using Science Principles 
Source: TIMSS 1999, Grade 8. 
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The first two science practice categories—Identifying Science Principles and Using Science 
Principles—both require students to correctly state or recognize the science principles contained in 
the content statements. A difference between the categories is that Using Science Principles focuses 
on what makes science knowledge valuable or, in other words, its usefulness in making accurate 
predictions about phenomena and in explaining observations of the natural world in coherent ways 
(i.e, “knowing why”). Distinguishing between these two categories draws attention to differences 
in the depth and richness of individuals’ knowledge of the content statements. Certain actions on 
the part of students lead to an inference of Identifying Science Principles, while other actions lead 
to an inference of Using Science Principles. Assuming a continuum from “just knowing the facts” to 
“using science principles,’ there is considerable overlap at the boundary. The line between the identi- 
fying and using categories is not distinct. Consider the following item, which illustrates the expecta- 
tion that students will connect different representations. In this case, the student must identify the 


correct pictorial representation of a complete electrical circuit. 


Illustrative Item 


The pictures show a light bulb connected to a battery. 
Which bulb will light? 


Key: C 


P4.11, Using Science Principles 
Source: TIMSS 2003, Grade 4. 


Student responses to this item are open to two interpretations. If students have had a great deal 
of exposure to these types of circuit representations, their responses would fall under Identifying 
Science Principles. However, if these circuit representations are relatively new for students, then they 


would need to apply more reasoning and their responses would fall under Using Science Principles. 
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The following textbox provides further illustration of the distinction between identifying the 
boiling point of water (a fact) and using the relationship between boiling point and pressure (alti- 


tude) to explain or predict. 


Clarification: Distinguishing Between Identifying Science Principles 
and Using Science Principles—A Boiling Point Example 


Grade 8 Content Statement: Matter—Properties of Matter: 
P8.4: ...Each element and compound has physical and chemical properties, such as boiling point, 


density, color, and conductivity, which are independent of the amount of the sample. 


Distinguishing between the two categories of Identifying Science Principles and Using Science 
Principles is a function of actions or performances. Using boiling point as an example, one might 
observe different responses to the question, “What is the boiling point of water?” Behaviors or 


actions might include: 


e Penciling in the oval corresponding to 100 °C in a selected-response item. 

e Writing: “The boiling point of water is 100 °C at sea level.” 

e Writing: “The boiling point of water changes as pressure changes. So, even though water 
boils at 100 °C at sea level (1 atm pressure), it might boil at a lower temperature on top 


of a mountain because pressure is lower up there.” 


The above responses evoke different inferences about the science understanding of the individual 
responding. Both the first and second responses suggest that the question is only assessing knowl- 
edge of facts or the ability to identify a science principle; however, they illustrate the difference 
between recognizing a correct answer and retrieving that correct answer from memory. The third 
response contains even more sophisticated information, suggesting that the student can use a 


science principle to make predictions. Distinctions between these two categories can be clarified 


by examining student responses and conducting cognitive labs. 
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The following item set illustrates a combination of Identifying Science Principles and Using 
Science Principles. It also addresses the expectation that students interpret data presented in a graph 
and use the data to perform a mathematical calculation. 


Illustrative Item 


The graph below shows the distance traveled over time by a student walking down a hall. 
Use the information shown on the graph to do Numbers 7 and 8. 


Distance Traveled Over Time 


own © 


me NMYNH Ww 


Oo ws 


Distance (m) 


0 5 10 15 20 25 30 
Time (seconds) 


7. During which time interval was the student moving the fastest? 
OA 
OB 
OC 
OD 


Key: D 


P8.14, Identifying Scientific Principles 
8. What was the average speed of the student from 0 seconds to 5 seconds? 


Average speed: 


(See appendix C for item scoring guides.) 


P8.14, Using Scientific Principles 
Source: Colorado Department of Education 2002, Grade 8. 
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Using Scientific Inquiry 


Scientists make observations about the natural world, identify patterns in data, and propose 
explanations to account for the patterns. Although scientists differ greatly from one another in the 
phenomena they study and in how they go about their work, scientific inquiry involves the collec- 
tion of relevant data, the use of logical reasoning, and the application of imagination and evidence 
in devising hypotheses to explain patterns in data. Scientific inquiry is a complex and time-intensive 
process that is iterative rather than linear. Scientists are also expected to exhibit, indeed to model, 
the habits of mind—curiosity, openness to new ideas, informed skepticism—that are part of science 
literacy. This includes reading or listening critically to assertions in the media, deciding what evi- 
dence to pay attention to and what to dismiss, and distinguishing careful arguments from shoddy 
ones. These critical-thinking and systems-thinking skills are the basis for exercising sound reasoning, 
making complex choices, and understanding the interconnections among systems (Partnership for 
21st Century Skills 2004). Thus, using scientific inquiry depends on the practices described above— 
Identifying Science Principles and Using Science Principles. Moreover, in addition to involving 
declarative and schematic knowledge, Using Scientific Inquiry draws heavily on procedural knowl- 
edge (“knowing how, ¢.g., knowing how to determine the mass of an object). This framework 
focuses on a few key inquiry practices that are practical to measure in the NAEP Science Assess- 


ment. Using Scientific Inquiry comprises the following general types of performance expectations:* 


e Design or critique aspects of scientific investigations (e.g., involvement of control groups, 
adequacy of sample). 

e Conduct scientific investigations using appropriate tools and techniques (e.g., selecting 
an instrument that measures the desired quantity—length, volume, weight, time interval, 
temperature—with the appropriate level of precision). 

e Identify patterns in data and/or relate patterns in data to theoretical models. 

e Use empirical evidence to validate or criticize conclusions about explanations and pre- 
dictions (e.g., check to see that the premises of the argument are explicit, notice when the 


conclusions do not follow logically from the evidence presented). 


Scientific inquiry is more complex than simply making, summarizing, and explaining observa- 
tions, and it is more flexible than the rigid set of steps often referred to as the “scientific method.” 
The National Standards make it clear that inquiry goes beyond “science as a process” to include an 


understanding of the nature of science (p. 105) and further state the following: 
It is part of scientific inquiry to evaluate the results of scientific investigations, exper- 


iments, observations, theoretical models, and the explanations proposed by other scien- 


tists. Evaluation includes reviewing the experimental procedures, examining the evidence, 


8 Tn addition, 12th graders at the Advanced level are expected to be able to identify a scientific question for investigation. See appendix B for achieve- 
ment level descriptions. 
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identifying faulty reasoning, pointing out statements that go beyond the evidence, and 


suggesting alternative explanations for the same observations (p. 171). 


In the NAEP Science Assessment, when students use scientific inquiry they are drawing on their 


understanding about the nature of science, including the following ideas (see Benchmarks): 


e Arguments are flawed when fact and opinion are intermingled, or the conclusions do not 
follow logically from the evidence presented. 

e A single example can never support the inference that something is always true, but some- 
times a single example can support the inference that something is not always true. 

e Ifmore than one variable changes at the same time in an experiment, the outcome of the 
experiment may not be clearly attributable to any one of the variables. 

e The way in which a sample is drawn affects how well it represents the population of 
interest. The larger the sample, the smaller the error in inference to the population. How- 
ever, large samples do not necessarily guarantee representation, especially in the absence of 
random sampling. 


NAEP will assess students’ abilities to use scientific inquiry in two ways: students will be 
required to do the practices specified above and students will critique examples of scientific inquiry. 
It is incorrect to assume that assessment of using scientific inquiry is best or only achieved through 
hands-on performance tasks and interactive computer tasks (ICTs). In both cases of doing and 
critiquing, some assessment tasks will also be presented as traditional items in a digital platform. 

In doing, tasks may present data tables and ask students which conclusions are consistent with the 
data. Other tasks will be presented as hands-on performance and/or interactive computer tasks 

(e.g., students collect data and present their results, or students specify experimental conditions on 
computer simulations and observe the outcomes). In regard to critiquing, students might be asked 
to identify flaws in a poorly designed investigation or suggest changes in the design to produce more 
reliable data. Tasks may be based on print or electronic media (e.g., items may ask students to sug- 
gest alternative interpretations of data described in a newspaper article). Chapter four contains more 


information on types of items. 
The following middle school (grade 8) item illustrates the expectation that students will con- 


duct scientific investigations. By manipulating the simulation, students gather data and solve the 
problems that are presented. 
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Illustrative Item 


This interactive computer task is one module in an extended assessment of students’ abilities to 
use a range of technologies to investigate a complex problem, “Should lynx be reintroduced into 
a national park?” Students accessed, organized, and analyzed data on the number of hares in the 
park over a 25-year period, researched factors that would impact the population, and created a 


graph to analyze the trend. (See appendix C for a description of the full task.) 


This module allows students to interact with a simulated predator/prey (lynx/hare) population 
model. Students use the modeling tool to observe population trends that result from different 
parameter values for the lynx and hare populations. The screen shot below is an example of what 
students see after they have selected parameters and run the simulation. Note that it is a single 
screen shot and represents only a small subset of the many screens that students actually see when 
engaged in this interactive computer task. After students have run the modeling software, they are 


asked a series of questions (e.g., size of the hare population over time). 


CE = = acl 


Initial Number of Hares 


SS 

low high 
Initial Number of Lynx 
=i) 

low high 


f 
gol 


29 
time (years) 


L8.6, Using Scientific Inquiry 
Source: Quellmalz et al. 2004. 
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Using Technological Design 


In both the National Standards and Benchmarks, the term “technological design”? refers to the 
process that underlies the development of all technologies, from paper clips to space stations. The 
National Standards explain that this meaning “is not to be confused with ‘instructional technology; 
which provides students and teachers with exciting tools—such as computers—to conduct inquiry 
and to understand science” (p. 24). 


In the framework, using technological design describes the systematic process of applying sci- 
ence knowledge and skills to solve problems in a real-world context. The National Standards clearly 
state the reason for including technological design in the science curriculum: “Although these are 
science education standards, the relationship between science and technology is so close that any 
presentation of science without developing an understanding of technology would portray an inac- 
curate picture of science” (p. 190). The National Standards define technology and its relationship to 


science as follows: 


As used in the Standards, the central distinguishing characteristic between science and 
technology is a difference in goal: The goal of science is to understand the natural world, 
and the goal of technology is to make modifications in the world to meet human needs. 


Technology as design is included in the Standards as parallel to science as inquiry (p. 24). 


As in scientific inquiry, the professional practice of technological design (also called engineering 
design) is complex and time intensive. Because NAEP addresses the subject area of science, the use 
of technological design components in the NAEP Science Assessment will be limited to those that 
reveal students’ abilities to apply science principles in the context of technological design. Students’ 
abilities to Identify and Use Science Principles should provide the opportunities as well as the limits 
for assessment tasks related to Using Technological Design. For example, if students are asked to 
design a town’s energy plan, they may be expected to consider the environmental effects of using 
natural gas versus using coal, but they would not be expected to consider the economic, political, or 


social ramifications of such a plan. 
The framework samples key components of Using Technological Design from the more com- 
plete descriptions found in the National Standards and Benchmarks. Using Technological Design 


comprises the following general types of performance expectations, all of which involve students 


using science knowledge to accomplish the following: 


° This practice is discussed in some detail because it is new in NAEP Science Assessments. 
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e Propose or critique solutions to problems, given criteria and scientific constraints. 
e Identify scientific tradeoffs in design decisions and choose among alternative solutions. 


e Apply science principles or data to anticipate effects of technological design decisions. 


The three components of Using Technological Design are discussed below. 


First, the technological design process is rooted in the definition of a problem that can be solved 
through a technological design process. The problem generally describes a human need or want and 
specifies criteria and constraints for an acceptable solution (International Technology Education 
Association 2000). Only constraints that reflect the science content statements in this framework 
will be considered in developing relevant NAEP assessment items. The engineer who designs a 
bridge, for example, must take into account the effects of wind and water currents by using relevant 


physics principles to simulate these effects on possible structures before the bridge is built. 


Second, even if limited to the application of science principles, choosing between alternative 


solutions almost always involves tradeoffs. As stated in the Benchmarks: 


There is no perfect design. Designs that are best in one respect ... may be inferior in 
other ways. ... Usually some features must be sacrificed to get others. How such trade-offs are 


received depends upon which features are emphasized and which are down-played (p. 49). 


The application of science principles may be used to compare alternative technological solutions 


to see which will better solve the problem and accomplish the project’s goals. 


Finally, while the chosen solution may be intended to solve a human problem or meet a human 
need, the effects are not always as planned. When the automobile was invented, no one could have 
predicted the environmental and human health impacts of vehicle emissions. However, scientists 
and engineers must work together to apply their knowledge of the natural world to make such 
predictions. According to the National Standards, students in grades K—4 should know about the 


effects of design solutions: 


People continue inventing new ways of doing things, solving problems, and getting 
work done. New ideas and inventions often affect other people; sometimes the effects are 
good and sometimes they are bad. It is helpful to try to determine in advance how ideas and 


inventions will affect other people (p. 140). 
In terms of cognitive demands, both declarative (knowing that) knowledge and schematic 


(knowing why) knowledge come into play for the three components of Using Technological Design, 
as does strategic knowledge (knowing when and where to apply knowledge). 
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The role of technological design in US. science classrooms currently varies widely, and it is not 
possible to predict the extent to which it will be integrated into the school curriculum in the future. 
The role of technological design in the NAEP Science Assessment will need to be revisited regularly 
in response to its evolving role in school science. 


Since using technological design in the NAEP Science Assessment must have direct relevance to 
science, it is assumed that students have some understanding of the relationship between science and 


technology. The science-technology relationship is further discussed in chapter four as providing 
context for assessment items. 


The following item illustrates the expectation that students will apply science principles to antic- 
ipate the effects of a technological design decision. 
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Illustrative Item 


Occasionally, a fire will destroy a forest, burning down trees and pushing wildlife out of their 
forest homes. However, the forest will grow back. Eventually, through the process of forest succes- 


sion as shown below, short grasses and flowers begin to grow and animals make new homes. 


Forest Succession 


Shrubs trees 


Tall 
Short grasses 
grasses Time 


Over time, shrubs and trees begin to grow. The forest returns to a lush habitat for the wildlife 
listed in the chart below. 


Forest Wildlife 


Ground dwelling Worms, beetles 


Reptiles and amphibians | American toads, wood frogs, snakes, Eastern box turtles 


Small animals Squirrels, chipmunks 


Medium to large animals | Opossums, raccoons, white-tailed deer, black bears 


Airborne Butterflies, moths, bees, wild turkeys, red-tailed hawks, bald eagles 


A power company owns part of a forest that was destroyed by a fire. The forest could take decades 
to rebuild on its own. The company’s department of environmental studies suggests planting new 
trees to help the forest rebuild. 


Using the information in the scenario: 


e Explain how planting trees could benefit the natural ecosystem. 


e Explain how planting trees could harm the natural ecosystem. 


(See appendix C for item scoring guides.) 


E8.15, Using Technological Design 
Source: Washington Assessment of Student Learning 2004, Grade 8. 
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Summary of Practices 


The general performance expectations for each of the four practices are summarized in exhibit 


13. Dashed lines indicate that the boundaries between these categories are not distinct, and some 


overlap is to be expected. 


Exhibit 13. General performance expectations for science practices 


€ Communicate accurately and effectively > 


Identifying 
Science 
Principles 


Using 
Science 
Principles 


Using 
Scientific 
Inquiry 


Identify scientific tradeofts 
in design decisions and 


Using 
Technological 
Design 


Describe, mea- 
sure, or classify 
observations. 


Explain obser- 
vations of 
phenomena. 


Design or critique 
aspects of scientific 
investigations. 


Propose or critique 
solutions to prob- 
lems given criteria 
and scientific 
constraints. 


State or recognize 
correct science 
principles. 


Predict obser- 
vations of 
phenomena. 


Conduct scientific 
investigations 
using appropri- 
ate tools and 
techniques. 


choose among alternative 


solutions. 


Demonstrate rela- 
tionships among 
closely related 
science principles. 


Suggest examples 
of observations 
that illustrate a 
science principle. 


Identify patterns 
in data and/or 
relate patterns in 
data to theoretical 
models. 


Demonstrate 
relationships 
among different 
representations of 
principles. 


Propose, analyze, 
and/or evalu- 

ate alternative 
explanations or 
predictions. 


Use empirical 
evidence to val- 
idate or criticize 
conclusions about 
explanations and 
predictions. 


Apply science principles 
or data to anticipate effects 
of technological design 


decisions. 
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Clarification: Sample Performance Expectations for a Life Science Content Statement 


The examples below are all related to the following grade 8 Life Science content statement: 


L8.4: Plants are producers—they use the energy from light to make sugar molecules from the 
atoms of carbon dioxide and water. Plants use these sugars along with minerals from the soil to 
form fats, proteins, and carbohydrates. These products can be used immediately, incorporated 


into the plant’s cells as the plant grows, or stored for later use. 
All examples are also related to a specific situation: 


Two different varieties of grass—one better adapted to full sunlight and one better adapted to 


shade—are each grown in sunlight and in shade. 


The results of a controlled experiment along these lines might resemble the following: 


Condition Grass Type A Grass Type B 
Sunlight Better growth* Less good growth* 
Shade Less good growth* Better growth* 


* Several variables could be used to indicate growth: mass or dry mass of plants, thickness of 


stems, number of new sprouts, etc. 


Identifying Science Principles 


1. State from where a plant’s food originates. 


2. Classify the grass plants as producers or consumers. 


The first performance calls for students to repeat information found in the content statement 
with little or no modification. The second performance asks students to use the definition of 


producers given in the content statement to classify or identify the plants. 


Using Science Principles 


1. Predict whether sugar will move up or down the stems of the grass plants and explain your 
prediction. 


2. Explain where the mass of the growing grass originates. 


These performances require students to use principles in the content statement to predict or 
explain specific observations (growing grass in this case). The content statement itself does not 


provide the answers to the questions. 
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Using Scientific Inquiry 


1. Given a data table showing the mass of grass plants of each type grown in the sunlight and 
shade, draw conclusions about which variety of grass is better adapted to each condition. 
2. List other variables that should be controlled in order to feel confident about your 


conclusions. 


The first performance is related to the content statement in that the importance of light for 
plant growth is useful background information for students. However, the performance requires 
interpretation of new information (the data table) that has to do with differences among types 
of plants, while the content statement contains generalizations about all plants. Thus, the per- 
formance requires students to use the data to develop new knowledge that they did not have 
previously. The second performance is in part an assessment of the students’ understanding of 
experimental design. However, good answers would also require knowledge of this and related 


content statements to identify variables that are relevant to plant growth. 


Using Technological Design 


1. Given experimental results on the growth of different varieties of grass plants under sun- 
light and shade conditions, develop a plan for using different types of grass seed in differ- 
ent parts ofa partially shaded park. 


This performance requires students to use knowledge of the content statement and the experi- 
mental results to accomplish a practical goal (in this case, a park with grass growing well in areas 


that receive varying amounts of sunlight). 


Performance Expectations 


The NAEP Science Assessment will focus on how students bring science content (as described 
in chapter two) to bear as they engage in the practices described in this chapter. That is, science 
practices are not content-free skills; they require knowledge of the Physical, Life, and Earth and 
Space Sciences as well as knowledge about scientific inquiry and the nature of science (e.g., drawing 


conclusions from investigations). 


Performance expectations are derived from the intersection of content statements and science 
practices. If the content statements from the Physical, Life, and Earth and Space Sciences are the 
columns of a table and the practices (Identifying Science Principles, Using Science Principles, Using 
Scientific Inquiry, Using Technological Design) are the rows, the cells of the table are inhabited 
by performance expectations. Examples are provided in exhibit 14, which is based on exhibit 1 in 
chapter one. Note that performance expectation cells may overlap because the content and practice 


categories themselves are not distinct (as indicated by dashed lines). 
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Exhibit 14. Generating examples of grade 8 performance expectations 


Science Content 


Physical Science 
Content Statements 


Life Science 
Content Statements 


Earth and Space 
Sciences 
Content Statements 


| Identify the units Identify the raw mate- ! Identify wind as the 
Identifying ; that might be used to rials that plants use to —-- movement of air from 
Science ' measure the speed ofan: make sugars (see L8.4). | higher to lower pressure 
Principles; antandthespeedofan ; : regions (see E8.11). 
| airplane (see P8.14).* | 
| An object (e.g., a toy car) Explain why sugars are | Explain why mountain 
' moves with aconstant | foundto move primar- | soils are generally thin- 
; speed along a straight : ily down the stem of : ner than floodplain soils 
Using Science | line. Predict (with ‘ a growing plant (e.g., | (see E8.6). 
Principles ' justification) what might ! potato, carrot) (see 
' happen to this object’s + L8.4). 
' speed as it rolls downhill : 
3 ' (see P8.16). | 
0 ---------.------. --- SSRIS 
i I 2 isos Metues Yee ed ound | ucutsucuunerwbedgeieevaeeuapwaddcenees oven Suudedcewens’ 
2 ' Design an experiment —_: Criticize conclusions | Given data (indexed 
§ ' todetermine howthe | about likely conse- ' by month) on annual 
a ; speed of a battery-oper- quences of consuming | trends of incoming solar 
Using ' ated toy car changes asa : various dietsbasedon __: radiation for five cities, 
Scientific | result of added mass (see } flawed premises or flaws | determine whether 
Inquiry : P8.16). : in reasoning (see L8.5). | the location is in the 
‘ Northern or Southern 
Hemisphere 
(see E8.12). 
Evaluate the following Identify possible ecolog- Describe the conse- 
; ' car designs to determine ‘ ical side effects of agri- | quences (e.g., erosion) 
Using =e ; ; is | ; 
; : which one is most likely ‘ cultural fertilizer runoff ‘ of undercutting a steep 
Technological | Scat Ce 
p ' tomaintainaconstant : intoalake (see L8.7). _! slope fora road cut (see 
Design |; . H 
: speed asitgoesdowna | | E8.4). 
| hill (see P8.16). 


* To identify the science content statement on which each performance expectation is based, the content 
statement’s unique code (from exhibits 8, 10, and 12 in chapter two) is provided. 
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The content statements from chapter two on which these performance expectations are based 
are written in general terms. The process of creating performance expectations requires further clar- 
ification of the content statements themselves. As described in chapter two, this involves “detailing” 
the meanings of the content statements and setting boundaries on the content to be assessed at a 
given grade level. Moreover, if the crossing of content statements with practices were done for every 
science content statement and practice, the number of performance expectations generated could 
be unmanageably large. Exhibit 15 provides selected example performance expectations for a single 
Earth and Space Sciences content statement. The specifications include additional examples. These 
examples are illustrative, not exhaustive. It is expected that assessment developers will continue this 


process for all content and practices sampled for a particular NAEP Science Assessment. 


Performance expectations are written with particular verbs indicating the desired performance 
expected of the student. The action verbs associated with each practice are not firmly fixed, and the 
use of any action verb must be contextualized. For example, when the science practice component 
“conduct scientific investigations” is crossed with a states-of-matter content statement, it can gener- 
ate a performance expectation that employs a different action verb, such as “heats as a way to evapo- 


rate liquids.” 


Generating and Interpreting Items 


Neither the content statements from chapter two nor the practices discussed in this chapter will 
be assessed in isolation. All assessment items will be derived from a combination of the two (i.e., 
from performance expectations). Observed student responses to these items can then be compared 
with expected student responses to make inferences about what students know and can do. Exhibit 
15 is an Earth and Space Sciences example of the process of generating and interpreting items from 
performance expectations. The item examples are of two types: item suggestions (descriptions of 
items to be developed) and illustrative items (released items from various large-scale assessments). 
Additional examples of the process of generating and interpreting items are provided in the 


specifications. 
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Exhibit 15. Earth and Space Sciences example of generating and interpreting items 


Grade 8: Earth in Space and Time— Objects in the Universe 


Content Statement 


E8.2: Gravity is the force that keeps most objects in the solar system in regular and predictable motion. 
Those motions explain such phenomena as the day, the year, phases of the Moon, and eclipses. 


Commentary 
This content statement encompasses two interrelated sets of concepts: 


1. Gravity acts between and among all objects in the solar system, and it plays an essential role in the regu- 
lar and predictable motions of planets around the Sun and of satellites around planets. 


e On Earth, gravity is experienced as a force that pulls everything “down” toward the center of the 
Earth. (A common naive conception is that the atmosphere “pushes” things down and causes gravity.) 

e Gravity is a force of attraction that is exerted by every object on every other object. 

e Gravity exists in space and on other planets. (A common naive conception among students is that 
there is no gravity in space because space has no air.) 

e The almost circular motion of planets and satellites results from the force of gravity and the tendency 
of a body to continue moving through space in a straight line unless acted upon by a net force. 


2. The regular and predictable motions of Earth, the Sun, and the Moon cause the cyclic phenomena that 
can be observed in the sky. 


e The day-night cycle results from Earth’s rotation on its axis once in 24 hours. 

e Annual changes in the visible constellations and the seasons result from Earth’s revolution around the 
Sun once every 365% days. 

e Moon phases result from the Moon’s orbit around Earth about once a month, which changes the part 
of the Moon that is lighted by the Sun and how much of the lighted part can be seen from Earth. 


Note the connection between this content statement and the Physical Science subtopic, forces affecting 
motion. 


Students are not expected to use the inverse square relationship of gravitational force and distance to find 
the strength of the gravitational force between two objects. 


Students do not need to know that the motion of planets and satellites is elliptical and not circular. 


Examples of Performance Expectations 


Identifying Science Principles. Students can: 


e Identify gravity as the force exerted by every object in the solar system on every other object. 

e Identify gravity as the force that keeps the Moon circling Earth, rather than flying off into space. 

e Describe the regular motions of Earth through space, including its daily rotation on its axis and its 
yearly motion around the Sun. 
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Exhibit 15 (continued). Earth and Space Sciences 
example of generating and interpreting items 


Examples of Performance Expectations (cont.) 


Using Science Principles. Students can: 


e Explain that the orbit of one object around another is due to the tendency of an object to move in a 
straight line through space and due to the force of gravity between the two objects. 

e Explain how the monthly pattern of Moon phases observed from a point on Earth results from the 
Moon’s orbit around Earth, which changes the part of the Moon that is lighted by the Sun and what 
portion of the lighted part can be seen from Earth. 

e Distinguish between explanations for lunar (Moon) phases and lunar eclipses. 

e Explain that astronauts and other objects in orbit seem to “float” because they are in free fall, under 
the influence of gravity. 


Using Scientific Inquiry. Students can: 


e Arrange a set of photographs of the Moon taken over a month’s time in chronological order and 
explain the order in terms of a model of the Earth-Sun-Moon system. 

e Design a plan for observing the Sun over a year’s time to find out how the length of the day is related 
to the rising and setting point of the Sun on the horizon. 

e Design a series of observations or measurements to determine why some objects—such as certain 
asteroids or comets—visit the solar system just once, never to return. 


Using Technological Design. Students can: 


e Choose among several (qualitative) methods for aiming a rocket so that it reaches the planet Mars and 
give a rationale that shows understanding of orbital motion. 

e Use scientific tradeoffs in deciding whether or not to support a plan to observe and predict orbits of 
asteroids that enter the inner solar system. 

e Given a scenario in which a person is shipwrecked on an island in the ocean, critique plans to create a 
calendar to keep track of the passage of time. 


Items To Assess Identifying Science Principles 


Illustrative Item 1 


What force keeps the planets in our solar system in orbit around the Sun? 


A. gravitational 
B. magnetic 

C. electrical 

D. nuclear 


Source: Adapted from Massachusetts Department of Education, 
Massachusetts Comprehensive Assessment System (MCAS) 2000, Grade 8.* 


*MCAS materials appearing in the framework have been released to the public by the Massachusetts Depart- 


ment of Education and are available at no cost at http://www.doe.mass.edu/mcas/testitems.html. 
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Exhibit 15 (continued). Earth and Space Sciences 
example of generating and interpreting items 


Items To Assess Identifying Science Principles (cont.) 


Illustrative Item 2 


The drawings show a rocket being launched from Earth and returning. 


Position 2 
Position 3 


Position 1 


In which of these positions does gravity act on the rocket? 


A. position 3 only 

B. positions 1 and 2 only 
C. positions 2 and 3 only 
D. positions 1, 2, and 3 


Key: D Source: TIMSS 1999, Grade 8. 


Items To Assess Using Science Principles 


Illustrative Item 


A space station is to be located between the Earth and the Moon at the place where the Earth’s gravi- 
tational pull is equal to the Moons gravitational pull. 


On the diagram below, circle the letter indicating the approximate location of the space station. 


Earth 


Moon 


Explain your answer. 


Source: NAEP 1996, Grade 8. 
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Exhibit 15 (continued). Earth and Space Sciences 
example of generating and interpreting items 


Items To Assess Using Science Principles (cont.) 


Interpretation: The correct answer is C. Since the Moon has one-sixth the amount of gravity as Earth, 

a body that experiences an equal gravitational force from Earth and the Moon should be closer to the 
Moon. Point C is the only point that is closer to the Moon. Note: Point C is about one-twelfth of the way 
between the Moon and Earth; it should be one-sixth of the distance. (See appendix C for the item scoring 
guide.) 


Item Suggestion 
Is there gravity in space? Which of the following gives the best response to this question? 


A. No. You can see that astronauts float around weightless in their cabin. 

B. No. There is no air in space, so gravity cannot exist there. 

C. Yes. There must be gravity since planets keep circling the Sun. 

D. Some. The Moon has one-sixth as much gravity as Earth, so we know there is some gravity in 
space. 


Key: C 

Interpretation: The correct answer is C. This question is drawn from a series of studies that show a com- 
mon naive conception—that there is no gravity in space because space has no air. The distractors are drawn 
from student interviews. It is likely that these naive conceptions stem from images that students have seen 


of astronauts floating around in a “weightless” environment while in orbit. This item probes schematic 
knowledge (see Cognitive Demands section later in this chapter). 


Item To Assess Using Scientific Inquiry 


Item Suggestion 
A student is presented with a set of photographs of the Moon taken over a month’s time. The photos 
are not presented in chronological order. The student is asked to arrange them in the order in which 
they were taken and explain the reason for Moon phases. 


Interpretation: This suggestion reflects items used frequently in curricular materials (e.g., Schatz and 
Cooper 1994). Students are asked to find patterns in the data. First, they should be sufficiently familiar 
with the lunar cycle to arrange the photographs in order, either in a line to represent a chronology or 
in a circle to represent a cycle (tapping declarative knowledge and procedural knowledge to a lesser 
extent—see Cognitive Demands section later in this chapter). Then, students should be able to explain 
Moon phases in terms of the Moon circling Earth and the changing angle between the Sun and Moon 
as observed from Earth. This part of the item probes schematic knowledge—see Cognitive Demands 
section later in this chapter. This is a challenging question that many educated adults fail to answer. 
However, studies show that middle school students can learn to do this exercise by observing lunar 
phases and explaining them using a model of the Earth-Sun-Moon system (Barnett and Morran 2002; 
Kavanagh, Agan, and Sneider 2005). 
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Exhibit 15 (continued). Earth and Space Sciences 
example of generating and interpreting items 


Items To Assess Using Technological Design 


Item Suggestion 1 


NASA wants to launch a spacecraft with rockets from Earth so that it will reach and orbit Mars. 
Which of the following statements about this flight is WRONG? 


A. In the first phase of its flight, the forces acting on the spacecraft are the thrust of the rocket 
engine, gravity, and friction from the Earth’s atmosphere. 
B. When the rocket engine shuts off, the only force acting on the spacecraft is the force of gravity. 
C. Once the spacecraft is above the Earth’s atmosphere and the rocket engine is off, it will travel at a 
constant speed since there is no gravity in space. 
D. If the spacecraft is aimed correctly and has the proper speed, the spacecraft will reach Mars and 
require only engine braking to attain orbit. 
Key: C 
Interpretation: The correct answer is C because there is gravity in space and planning for such a rocket 
flight would need to take into account the gravity from Earth, Mars, and the Sun (declarative knowl- 
edge—see Cognitive Demands section later in this chapter). This question is drawn from a series of studies 
that show the following naive conceptions about gravity are common among many students at the middle 
school, high school, and even college levels: If a body is moving, there is a force acting on it in the direc- 
tion of motion (Finegold and Gorsky 1991; Gunstone and Watts 1985; Sequeira and Leite 1991); there 
is no gravity in space (Bar et al. 1994; Chandler 1991; Morrison 1999); and gravity cannot act in space 
because there is no air in space (Bar and Zinn 1998). One study showed that, with effective instruction, 
middle school students can overcome these naive conceptions and learn that gravity does, in fact, act in 
space, where it keeps satellites and planets in their orbits (Bar, Sneider, and Martimbeau 1997). 


Item Suggestion 2 


Decisions about whether or not to develop new technologies always concern tradeoffs. For example, most 
scientists today believe that the extinction of dinosaurs and many other species was caused by the collision 
of a large asteroid with Earth 65 million years ago. As a result, there is a proposal to develop two new 
technologies: (1) the detection and tracking of all asteroids large enough to do considerable damage if they 
should strike Earth, and (2) the development of a means to send a spacecraft to meet the asteroid in space 
and change its path. Write a paragraph describing: 


a. whether or not you think it would be possible to develop these technologies based on your 
knowledge of science; and 

b. some of the scientific tradeoffs that should be considered in deciding whether or not to develop 
these new technologies. 


Interpretation: Look for evidence that the students understand what asteroids are, that scientists have 
observed asteroids, and that observations taken at several points in time allow for the prediction of an 
asteroid’s path. Also look for evidence that students understand that spacecraft can be built, launched, and 
navigated to intercept solar system bodies. (In fact, several spacecraft have already intercepted asteroids 
and comets.) Regarding scientific tradeoffs, look for evidence that students recognize the advantages of the 
proposed technologies (e.g., avoid a catastrophic collision in which billions of people and animals could 
die) as well as possible negative effects (e.g., breakup of the asteroid so there are many collisions rather than 
one, accidents on launch). 
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Learning Progressions 


A learning progression is a sequence of successively more complex ways of reasoning about a 

set of ideas. For any important set of ideas in science, understanding increases over time as students 
learn more, moving from initially naive knowledge of the natural world to increasingly more sophis- 
ticated knowledge and conceptual understanding; this typically occurs in conjunction with educa- 
tional experiences in and out of school (NRC 2001). In other words, the progression from novice 
learner to competent learner to expert begins with the acquisition of relevant experiences, principles, 
concepts, facts, and skills and moves to the accumulation and organization of knowledge in a spe- 
cific domain and finally to expertise after extensive experience and practice (e.g., Ericsson 2002). The 


attention paid to growth of understanding may yield rich information about student progress. 


Research has been conducted on students’ learning progressions in some areas of science and at 
some levels of students’ development. It is expected that this research will directly inform the devel- 
opment of assessment items at grades 4, 8, and 12. For example, NRC has commissioned papers on 
learning progressions in evolution (Catley, Lehrer, and Reiser 2005) and in atomic molecular theory 
(Smith et al. 2004). Learning progressions provide opportunities for assessing specific content in 


greater depth. 


Several caveats about learning progressions are in order. First, learning progressions are not 
developmentally inevitable but depend on instruction interacting with the student’s prior knowl- 
edge and construction of new knowledge. Thus, learning progressions need to invoke assumptions 
about instruction. Second, there is no single “correct order”; there may be multiple pathways by 
which certain understandings can be reached. The particular pathway taken may be influenced 
by prior instructional experiences, individual differences, and current instruction (NRC 1999c, 
2001). Thus, learning progressions will necessarily be complex, involving multiple specific paths at 
the micro level, and will need to be described in ways that encompass such diversity. Third, actual 
learning is more like ecological succession with changes taking place simultaneously in multiple 
interconnected ways. Thus, attempts to describe specific sequences of learning performance (includ- 
ing those in the Catley, Lehrer, and Reiser (2005) and Smith et al. (2004) papers) must inevitably be 
artificially constrained and ordered. Finally, the learning progressions suggested in the framework 
and specifications are partly hypothetical or inferential because long-term longitudinal accounts of 


learning by individual students do not exist. 


Exhibit 16 uses a set of related science content statements across grades 4, 8, and 12 and follows 
the format of the science practices defined in the framework. It illustrates how relevant research (e.g., 
Smith et al. 2004) can be used as an opportunity to assess content in greater depth—available research 
on student learning is used to inform the generation of related performance expectations across 


grades. The exhibit includes examples of performance expectations for a possible learning progression 
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for states of matter. These illustrative performance expectations do not denote a sense of content pri- 


ority or importance and are not a complete representation of the research currently available. 


Exhibit 16. Examples of performance expectations for states of matter 


Grade 4 


(See content statement P4.3.) 


Grade 8 
(See content statement P8.1.) 


Grade 12 
(See content statement P12.1.) 


Identifying Science Principles 


Classify samples of material as 
solid, liquid, or gas. 


Identifying Science Principles 


Given an animation of molecules 
in motion, identify the substance 
that is being illustrated as a solid, 
liquid, or gas. 


Identifying Science Principles 


Explain why ice is harder than 
liquid water in terms of the 
strength of the force between the 
molecules. 


Using Science Principles 


Infer that a change of state (e.g., 
freezing or melting) affects the 
identity of an object but not the 
identity of the material of which 
it is made. 


Using Science Principles 


Predict how the mass of a sample 
of iodine will change after sub- 
limation. Justify the prediction 
based on what occurs during 
sublimation at a molecular level. 


Using Science Principles 


Use the concept of molecular 
arrangements and bonds to 
explain why graphite is very soft 
and diamond is very hard, even 
though they are both made of 


pure carbon. 


Using Scientific Inquiry 
Collect, display, and interpret 
data showing how the tempera- 
ture of a substance changes over 


time as it cools and becomes a 


solid. 


Using Scientific Inquiry 

Plan and conduct an investiga- 
tion to determine the melting 
point and boiling point of an 
unknown substance. 


Using Scientific Inquiry 

Using molecular theory, explain 
the results of experiments 
showing how the volume of three 
different liquids changes when 
they are heated. 


Using Technological Design 


Propose a method for determin- 
ing for certain whether holiday 
chocolates that have been shaped 
by different processes (melting, 
freezing, reshaping, or break- 
ing into pieces) have the same 
amount of chocolate in them. 


Using Technological Design 


Choose the best solution 

for increasing the altitude 

of a hot air balloon based 

on an understanding of the 
macroscopic and microscopic 
changes that occur when the gas 
inside the balloon is heated. 


Using Technological Design 


Design an instrument to measure 
temperature as accurately as pos- 
sible, taking into account both 
the thermal properties of liquids 
and solids to be used in the 
device and the structural shape 
and dimensions of the device. 


Cognitive Demands 


The four science practices (Identifying Science Principles, Using Science Principles, Using 


Scientific Inquiry, and Using Technological Design) articulate what students should know and be 


able to do with the science principles presented in chapter two. Certain ways of knowing and rea- 


soning (cognitive demands) underpin these four science practices. In this section, the four cognitive 
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demands—“knowing that,’ “knowing how,’ “knowing why, and “knowing when and where to apply 
knowledge”—are discussed briefly (see the Specifications for more information). The goal is to fur- 
ther clarify the descriptions of the science practices, to facilitate item specifications and item writing, 
and to provide a framework for interpreting students’ responses. That is, the set of four cognitive 
demands can be used as a lens to facilitate item development and to analyze student responses (Li 
and Shavelson 2001; Shavelson 2006; Shavelson, Ruiz-Primo, and Wiley 2005), thereby checking 
expectations regarding the science content and practice(s) that are being tapped by a given assess- 


ment item.'° 


“Knowing that” refers to declarative knowledge. This cognitive demand sets up the expectation 
that students should know and reason with basic science facts, concepts, and principles (e.g., den- 
sity is mass per unit volume) and that they should be able to recall, define, represent, use, and relate 
these basic principles as appropriate. This cognitive demand corresponds most closely to the science 


practice of Identifying Science Principles. 


“Knowing how” refers to procedural knowledge. This cognitive demand sets up the expectation 
that students can apply the science facts, concepts, and principles in doing science. For example, stu- 
dents should know how to perform simple (routine) and complex procedures such as systematically 
observing and recording which objects sink and float in water, using a balance scale, measuring an 
object’s mass, calculating an object’s density, and designing and interpreting the results of an investi- 
gation (e.g., manipulating one variable and holding others constant). Procedural knowledge under- 


lies much of the science practice of Using Scientific Inquiry as defined in this framework. 


“Knowing why” refers to schematic knowledge. This cognitive demand sets up the expectation 
that students can explain and predict natural phenomena as well as account for how and why scien- 
tific claims are evaluated, argued and justified, or warranted (explaining and reasoning with prin- 
ciples and models). That is, this cognitive demand deals with students’ understanding of how the 
natural world works (such as why some things sink and others float in water), why light is essential 
to the propagation of most plants, or why the Moon changes phases. This cognitive demand overlaps 
considerably with the science understanding expected in the practice of Using Science Principles 


and also in the practices of Using Scientific Inquiry and Using Technological Design. 


The last cognitive demand, “knowing when and where to apply knowledge,” or strategic 
knowledge, is commonly talked about as transfer of current knowledge to new situations (tasks or 
problems). Strategic knowledge involves knowing when and where to use science knowledge in a 
new situation avd reasoning through the novel task to reach a goal. Strategic knowledge sets up 


the expectation that students can take their current knowledge and apply it to a somewhat novel 


© More than one cognitive demand can be associated with the more complex science practices. These associations may shift according to the knowl- 
edge that students at different grade levels bring to an assessment task. 
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situation. Such adaptation of knowledge to a particular problem and context underlies especially the 


practices of Using Scientific Inquiry and Using Technological Design. 


The cognitive demands are related, not independent (similar to the science practices). That is, 
when explaining “why,” a student will need to call on “knowing that”; at times, in justifying “why,” 
a student may have to call on “knowing how.” In addition, depending on the novelty of the task, 
strategic knowledge (knowing when and where to apply knowledge) may be called into play. Nev- 
ertheless, these related cognitive demands can be distinguished, and it is helpful to do so for item 


development and interpretation of student responses. 
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CHAPTER FOUR 


OVERVIEW OF THE ASSESSMENT DESIGN 


This chapter provides an overview of the Specifications. It begins with a brief description of the 
NAEP Science Assessment and a discussion of how items can be set in certain contexts (e.g., history 
and nature of science) to illustrate components of science content that are not otherwise incorpo- 
rated in the content statements. The types of items to be included in the assessment are described 
and examples are provided in illustrative item textboxes. Answers to selected-response items are 
indicated in the textbox; scoring guides for constructed-response items are provided in appendix C. 
Although items in the textboxes may assess more than one content statement or practice, only the 
primary content and practice designations are provided. This follows NAEP practice, which uses 
only primary designations for items in the analysis and reporting of student responses. To capture 
the wide range of science content statements and practices, the assessment will contain an array of 
item types. Consideration is given to English language learners and students with disabilities. The 


chapter concludes with recommendations for small-scale special studies. 


Overview of the Science Assessment 


The NAEP Science Assessment will include items sampled from the domain of science achieve- 
ment identified by the intersection of the content areas and science practices (i.e., performance 
expectations) at grades 4, 8, and 12. The assessment will include selected-response (multiple-choice) 
items and constructed-response items (which include short and extended constructed-response 
items as well as concept-mapping tasks). Some combination items may require more than one 
response; they include item clusters, POE item sets, hands-on performance tasks, and interactive 
computer tasks. The responses requested may be all selected-response, all constructed-response, or a 
mixture, and in 2019, all responses were submitted in a digital platform. At each of grades 4, 8, and 
12, student assessment time will be divided evenly between selected-response items and construct- 
ed-response items. Extra assessment time will be provided for a portion of the student sample so that 


hands-on performance tasks and interactive computer tasks can be administered. 


At grade 4, the items will be distributed approximately evenly among Physical Science, Life 


Science, and Earth and Space Sciences. At grade 8, there is a somewhat greater emphasis on Earth 
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and Space Sciences. At grade 12, the balance shifts toward Physical Science and Life Science, with 


less emphasis on Earth and Space Sciences. 


Finally, the distribution of items across the science practices will be approximately 60 percent 
combining Identifying Science Principles and Using Science Principles, 30 percent Using Scientific 
Inquiry, and 10 percent Using Technological Design. From grade 4 to 8 to 12, the emphasis on 
Using Science Principles increases and the emphasis on Identifying Science Principles decreases. As 
students progress through the grades, it is expected that their critical response skills and method- 


ological and analytical capabilities will increase. 


Assessment Item Contexts 


Science-literate citizens should be familiar with certain components of science content, such 
as the history and nature of science and the relationship between science and technology. Chapter 
three addresses the nature of science largely through a discussion of the science practices (particu- 
larly Using Science Principles and Using Scientific Inquiry). The relationship between science and 
technology is partially addressed in a discussion of the Using Technological Design practice. The 
history and nature of science not only clarify facets of science practices but also the human aspect 
of science and the role science has played in various cultures. Students can see that science changes, 
and new conclusions can be reached based on new empirical data (e.g., development of the theory of 
plate tectonics, use of gaps in early versions of the Periodic Table to “discover” new elements). The 
reciprocal relationship between science and technology can be seen, for example, in that scientists 
use technological tools to empirically test proposed explanations for questions about the natural 
world and engineers develop adaptations to the natural world to address human problems, needs, 


and aspirations based in part on science. 


When items are written to reflect particular science content statements, they may be framed 
using the contextual components from the statements. Aspects of the history and nature of science 
and the relationship between science and technology should thus be incorporated into the contexts 


of assessment items as shown below. 
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Illustrative Item 


Ernest Rutherford found that when he fired alpha particles at a thin gold foil, some were 


scattered at large angles.* What caused this scattering? 


A. The gold’s positive atomic nuclei attracted the negatively charged alpha particles. 
B. The gold’s negative atomic nuclei repelled the negatively charged alpha particles. 
C. The gold’s negative atomic nuclei attracted the positively charged alpha particles. 
D. The gold’s positive atomic nuclei repelled the positively charged alpha particles. 


Key: D 
P12.2, Identifying Science Principles 


* The framework’s developers created this item for illustrative purposes; it requires further devel- 


opment in that it is not field tested or published. 


Types of Items 


The judicious selection of items lies at the heart of any effective assessment of science achieve- 
ment. The framework for the 1996-2005 NAEP Science Assessments called for multiple-choice 
items (selected response), open-ended items (constructed response), and performance exercises. 
Multiple-choice items made up about 40 percent of the assessment, as measured by student response 
time, and open-response items made up about 60 percent of assessment time. In addition, subsets of 
the students sampled were given an extra 20 minutes in grade 4 and 30 minutes in grades 8 and 12 
to complete hands-on performance tasks. This framework generally follows the 1996-2005 recom- 
mendations in item structure but also specifies additional item types—some selected response and 
others constructed response. Further, in 2019, all item types are presented to students in a digital 
platform. As noted above, the item distribution by item format in this framework, in terms of stu- 


dent response time, is 50 percent selected response and 50 percent constructed response. 


Two additional considerations are the need to develop items that probe students’ ability to 
use communication skills and quantitative reasoning skills in science (see p. 68). Although this 
framework does not prescribe the amount of assessment time to be spent on items that require 
specific forms of communication or application of mathematics, it is expected that items requir- 
ing these skills will be represented at all three grade levels. Further details will be provided in the 


specifications. 


97 | 


Justification for Variation in Types of Items 


Issues of time and cost are paramount in any assessment. Accordingly, most of the item for- 
mats on the NAEP Science Assessment will be traditional selected response and short constructed 
response. However, some more complex items (e.g., hands-on performance tasks) should be part of 


any science assessment. 


Responses to complex items often correlate positively and cluster with responses to simpler 


items. However, complex items are recommended in the framework for the following reasons: 


e Items may correlate positively with one another, but they do not necessarily measure the 
same thing; that is, positive correlations can arise even when the cognitive demands of the 
assessment items vary. Research has shown that items vary in their cognitive demands for 
different kinds of knowledge and reasoning (e.g., Leighton 2004). 

e The NAEP Science Assessment signals the kinds of responses to tasks, problems, and exer- 
cises, along with the kinds of knowledge and reasoning, that should be expected of students 


as a result of what is taught in the science curriculum (consistent with the National Stan- 
dards and Benchmarks). 


For these reasons, the framework calls for a variety of item types for the NAEP Science Assess- 


ment (see the specifications for further details and explanation). 


Definitions of Types of Items 


The framework distinguishes selected-response from constructed-response item formats. For 
selected-response formats, students respond to a question by selecting the answer they believe to be 
most scientifically justifiable from a given set of alternatives. In contrast, with constructed-response 
formats, students respond to a question by “generating” or “constructing” a response. The construct- 
ed-response might be a single word, a short answer, an essay explanation, a summary of a laboratory 


investigation using concrete materials, or typed responses to a computer simulation. 

In addition to these two main item formats, there are combination items that generally require 
more than one response. These include item clusters, POE item sets, hands-on performance tasks, 
and interactive computer tasks. These combination items can use an all selected-response format, an 
all constructed-response format, or a mixture of these two main item formats. 


Following are the main types of items to be used on the NAEP Science Assessment: 


1. Selected response 


— Individual multiple-choice items 
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2. Constructed response 
— Short constructed-response items 
— Extended constructed-response items 


— Concept-mapping tasks 


3. Combination 
— Item clusters 
— POE item sets 
— Hands-on performance tasks 


— Interactive computer tasks 


Item clusters and POE item sets may use selected-response items, constructed-response items, or 
both. For example, a set of POE items might include a multiple-choice item in which students select 
a prediction and a short constructed-response item in which students are asked to write a justifica- 


tion for their prediction. 


Hands-on performance tasks and interactive computer tasks also may use selected-response 
items, constructed-response items, or both. In recording their answers, students may be asked to 
respond to both selected-response and constructed-response items. For example, 12th graders might 
be asked to manipulate a computer simulation of a chemical reaction, a multiple-choice question 
could ask students to choose the correct mass of a reaction product, and a short constructed-re- 


sponse question could ask students to describe how mass is conserved in chemical reactions. 


Selected-Response Items 


Selected-response items include individual multiple-choice items. These types of items are 


described below. 


Individual Multiple-Choice Items 


Selected-response items most often are in a multiple-choice format. Students read, reflect, and 
then select an answer from, for example, four alternatives provided. The alternatives include the 
most scientifically justifiable response (the answer) as well as three “distractors.” The distractors 
should appear plausible to students but should not be scientifically justifiable; when feasible, the 
distractors should also draw from current understanding about students’ mental models and learn- 
ing progressions. Whenever possible and especially when the focus is on using science principles 
or “knowing why” (schematic knowledge), naive conceptions, explanations, and predictions of the 


natural or manmade world should serve as distractors. 
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Of the following two multiple-choice items, both require identifying science principles and 
both tap the cognitive demand “knowing that” (declarative knowledge). The first item taps simple 
factual content. The second item taps more conceptually sophisticated content. For some students, 
if they cannot easily recall the science content, the second item may require the practice of using 


science principles (i.e., tapping more of “knowing why” than “knowing that”). 


Illustrative Item 


Air is made up of many gases. Which gas is found in the greatest amount? 


A. Nitrogen 

B. Oxygen 

C. Carbon dioxide 
D. Hydrogen 


Key: A 


E8.7, Identifying Science Principles 
Source: TIMSS 1995, Grade 8. 


Illustrative Item 


i. (piss <— Equator 


L208 


The diagram above shows a map of the world with the lines of 
latitude marked. Which of the following places marked on the map is 
most likely to have an average yearly temperature similar to location 9? 


O08 


yaw > 


Key: A 


E8.12, Identifying Science Principles 
Source: TIMSS 2003, Grade 8. 
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Constructed: -Response Items 


Constructed-response items include short constructed-response items, extended constructed-re- 


sponse items, and concept-mapping tasks. 


Short Constructed-Response Items 


This item type generally requires students to supply the correct word, phrase, or quantitative 
relationship in response to the question provided in the item; illustrate with a brief example; or 
write a concise explanation for a given situation or result. Thus, students must generate the relevant 
information rather than simply recognize the correct answer from a set of given alternatives, as in 


selected-response items. Following is an example of a short constructed-response item. 


Illustrative Item 


On a hot, humid day the air contains a lot of water vapor. What happens to the water vapor 


in the air when the air becomes very cold? 
(See appendix C for item scoring guides.) 


P4.6, Identifying Science Principles 
Source: TIMSS 2003, Grade 4. 


Extended Constructed-Response Items 


This item type is generally multidimensional; that is, it taps into multiple content statements, 
practices, and/or cognitive demands. These types of items can provide useful insight into students’ 
levels of conceptual understanding and reasoning. They can also be used to probe students’ ability to 
communicate in the sciences. Such items generally present a situation within or across content areas 
and require students to analyze the situation, choose and carry out an alternative plan for address- 
ing it, and interpret their response in light of the original situation. Students may also be given an 
opportunity to explain their responses, reasoning processes, or approach to a problem situation. 
However, care must be taken (especially with 4th graders and English language learners) to ensure 
that language ability is not confused with science ability. 


The following item involves reasoning with mental models (on carbon cycling) and thus 


attempts to probe the practice of Using Science Principles and taps into the cognitive demand of 


“knowing why.” 


101 | 


Illustrative Item 


The biosphere (living organisms), the lithosphere (rocks and soils of Earth’s crust), and the atmo- 
sphere are all involved in the cycling of carbon atoms. Describe the role that each plays in the 


carbon cycle. 
(See appendix C for item scoring guides.) 


E12.12, Using Science Principles 
Source: New Standards Spring Field Test 1999 for High School.* 


* The New Standards Spring Field Test 1999 for High School is the property of the University of 
Pittsburgh and the National Council on Education and the Economy (NCEE) and may not be 
used, reproduced, or distributed without the express written permission of the University of Pitts- 


burgh and NCEE. 


Concept-Mapping Tasks 


Concept-mapping tasks may be considered a complex item type because of the cognitive 
demands placed on students. Concept maps can be used as a reliable and valid assessment of stu- 
dents’ ability to make connections among science principles (Ruiz-Primo and Shavelson 1996a). 
Thus, concept-mapping tasks tap a science ability that is difficult to measure by other means. These 
tasks address the practice of Identifying Science Principles and the cognitive demand of “declarative 


knowledge,’ in particular the organization of this knowledge. 


In a concept-mapping task, students should be given a set of six to eight concept terms and be 
asked to construct a map linking pairs of terms with directed arrows. Students should label each 
arrow with a word or phrase that explains the relationship between a pair of concept terms. An 
arrow-linked pair of concept terms is called a proposition. Students’ concept maps can be evaluated 
as to the accuracy of propositions in their maps. The following textbox provides an illustrative set of 
instructions for a concept-mapping task and a sample student response. See the Specifications for 


information about scoring concept maps. 
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Illustrative Item 


Once students are familiar with how to construct concept maps, they might encounter a set of 
task instructions that resemble the following (adapted from Ruiz-Primo et al. 2001, p. 107): 


Examine the concept terms listed below. The terms selected focus on the topic [insert topic]. 


Construct a concept map using the terms provided below. Organize the terms in relation to one 
another in any way you want. Draw an arrow between the terms you think are related. Label the 


arrow using phrases or only one or two linking words. 


You can construct your map on the blank pages attached. When you finish your map, check that 
(1) all the lines have an arrow, (2) all the arrows have labels, (3) your concept map uses all the 


terms provided, and (4) your map shows what you know about [insert topic]. 


List of terms: [insert list of terms] 


An 11-year-old student constructed the following concept map using terms associated with 


(var is falling (nin 
—_ = 
water 

A 

contain comes from 


(jiu contain 
goes 
into rivers 


flow to 


Ma 


oceans }<— shines 
on 


the water cycle: 


E8.11, Identifying Science Principles 
Source: White and Gunstone 1992, p. 16. 
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Combination Items 


Combination items include item clusters, POE item sets, hands-on performance tasks, and 
interactive computer tasks. Combination items may consist of an all selected-response format, an all 


constructed-response format, or a mixture of both formats. 


Item Clusters 


The NAEP Science Assessment should include item clusters, and their development should be 
guided by current research on different forms of these items. In this type of item set, two or more 
items focus on an important idea or “mental model.” Hence, these items tap the practice of Using 
Science Principles and the cognitive demand of “knowing why.’ Where a rigorous body of research is 
available on students’ conceptions (as there is about the solar system), item clusters provide opportu- 
nities to assess students’ understanding of a particular key science principle at some depth. This type 
of item set can probe the conceptions and mental models that underlie students’ explanations of and 
reasoning about the natural world. For example, the following questions were part of a set of cluster 
selected-response items probing high school students’ mental models in astronomy (percentages of 


student responses to each option are given in parentheses): 


Illustrative Items 
What causes day and night? 


A. The earth spins on its axis. (66%) 

B. The earth moves around the Sun. (26%) 

C. Clouds block out the Sun’s light. (0%) 

D. The earth moves into and out of the Sun’s shadow. (3%) 
E. The Sun goes around the earth. (4%) 


Key: A 
E8.2, Using Science Principles 


The main reason for its being hotter in summer than in winter is: 


A. The earth’s distance from the Sun changes. (45%) 

B. The Sun is higher in the sky. (12%) 

C. The distance between the northern hemisphere and the Sun changes. (36%) 
D. Ocean currents carry warm water north. (3%) 
E 


An increase occurs in “greenhouse” gases. (3%) 
Key: B 
E8.12, Using Science Principles 
Source: Sadler 1998, pp. 274-276. 


104 


Another approach to probing students’ conceptions of the natural world is to develop a cluster of 
ordered multiple-choice items. These items track students’ performance along a learning progres- 
sion from naive understanding through more reasoned naive conceptions to full and scientifically 
justified understanding. In this approach, the progression is first described and then divided into 
levels so that multiple-choice items can be designed specifically to assess the performance level that 
a student (or group of students) has reached. As noted earlier, constructed-response formats are not 


precluded from use in item clusters. 
POE Item Sets 


POE item sets ask the student to predict, observe, and/or explain as follows: A situation is 
described; the student’s task is to provide a prediction for what will happen (sometimes with jus- 
tification) and/or to provide an explanation for what appears to be an anomaly. POE items tend 
to tap the practice of Using Science Principles and the cognitive demand of “knowing why” (sche- 
matic knowledge). For example, the following POE item was used with middle school students and 


focuses on prediction based on a mental model of buoyancy: 


Illustrative Item 


Rich cut block A into two unequal parts.* Part B is 2/3 of the original block A, and part C is 1/3 
of the original block A. Block A sinks in water. What will happen to B and C when placed in 


water? 


A. Both B and C will float. 
B. B will sink; C will float. 
C. B will subsurface float; C will float. 
D. Both B and C will sink. 


Key: D 


P8.4, Using Science Principles 
Source: Adapted from Shavelson 2006, and Shavelson, Ruiz-Primo, and Wiley 2005. 


* This item assumes that the entire block consists of a completely homogeneous material. The 
possibility that the block is made of heterogeneous material is unlikely to occur to middle school 


students. 
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As noted earlier, a POE item can take either a selected-response or a constructed-response 
format. In the selected-response format, students choose from a set of possible alternatives (based on 
known alternative mental models). In the constructed-response format, the student’s task is to write 
out (with justification) a prediction or an explanation. In the POE selected-response item above, 
students are asked to choose among a set of predictions for whether the parts of the block will sink 
or float. They can choose an answer from several alternatives that include naive conceptions. This 
item could be extended to further probe students’ mental models of buoyancy. Students could 
be asked to justify their answers by selecting from a number of possible explanations; again, these 
alternatives could include the correct explanation along with distractor explanations based on naive 
conceptions. Or, instead of using a multiple-choice format, students could be told that the full block 
sinks in water and be asked to write their prediction (with justification) as to what will happen when 
the two parts are placed in water. Or, students could observe a simulation or video of what happens 


to the full block and the two parts and then be asked to explain what they have observed. 


Hands-On Performance Tasks 


Using hands-on performance tasks, students manipulate selected physical objects and try to 
solve a scientific problem involving the objects. These exercises, if carefully designed, can probe stu- 
dents’ abilities to combine their science knowledge with the investigative skills that reflect the nature 
of science and inquiry. In large-scale assessments such as NAEP, uniform administration must be 
ensured. In the past, this has been accomplished through the use of standardized performance assess- 
ment kits, with each exercise proctored and scored by trained personnel. Special accommodations 


may be necessary for some students. 


A particularly cogent criticism of most hands-on performance tasks administered in large-scale 
assessments is that, rather than tap into students’ ability to inquire into a problem, typical perfor- 
mance assessments instead measure students’ ability to follow step-by-step instructions to arrive 
at the expected answer. Assessment developers are likely to create these “recipe” types of exercises 
because they must take into account the vast differences in students’ participation in science courses 
and their experiences. Given these differences, the absence of structure might produce unanticipated 
responses that could be problematic for the assessment either at the time the data are collected or 
when students’ performances are scored by raters. Although both the concern for structure to make 
large-scale assessment manageable and the criticism of highly structured performance tasks are well 
taken, there is evidence that valid performance exercises can be designed, developed, administered, 
and scored without encountering major problems (e.g., Ruiz-Primo and Shavelson 1996b; Shavel- 
son, Baxter, and Pine 1991). 


In designing hands-on performance tasks, it is important to keep the following considerations 


in mind: The degree to which students engage in some aspect of scientific inquiry depends on 
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who selects the problem to be studied, who selects the procedures to be carried out in tackling the 
problem, and who selects the answer. The NAEP assessment should provide students with a chal- 
lenging problem. However, students must have the opportunity to determine scientifically justifiable 
procedures for addressing the problem and arriving at a solution. Indeed, the problem to be solved 
involves setting forth procedures that manipulate the variable of interest, control extraneous vari- 
ables, and provide solid data to be used in arguing for and justifying a problem solution. In addition 
to allowing students to determine the procedures for carrying out the experiment, NAEP hands-on 
performance tasks should be “content rich” in that they require knowledge of science principles to 


carry them out. 


In brief, any hands-on performance task included in the NAEP assessment should present 
students with a concrete, well-contextualized task (problem, challenge); laboratory equipment and 
materials; and a response format that leaves the exercise process open. Students’ scores should be 
based on both the procedures created for carrying out the investigation and the solution (Shavelson, 
Baxter, and Pine 1991). The assessment, then, should provide a problem that draws on science prin- 
ciples and practices (performance expectation of interest) and leave students free to design and carry 
out the exercise to arrive at an answer or solution. The following item, designed for fifth graders, is 


an example of such a task. 


Illustrative Item 


Students are asked to identify the contents of each of the six boxes (A-F) by using the batteries, 
bulbs, and wires they are given to complete a circuit. This task requires knowledge of series cir- 
cuits but leaves problem-solving procedures up to the student. (See appendix C for more informa- 
tion about this task.) 


/ Electric Mystery Boxes“. 


i an 


P4.11, Using Scientific Inquiry 


Source: Shavelson, Baxter, and Pine 1991. 
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Interactive Computer Tasks (ICTs) 


The NAEP Science Assessment should include some but not necessarily all of the following four 
types of ICTs: (1) information search and analysis, (2) empirical investigation, (3) simulation, and 
(4) concept mapping. This framework uses static screen shots to illustrate examples of ICTs; how- 


ever, the screen shots represent only a small subset of the many screens students see when engaged in 
actual ICTs. 


Information search and analysis items pose a scientific problem and ask students to query an 
information database to bring conceptual and empirical information to bear (through analysis) on 


the problem. Following is a screen shot from such an ICT developed for eighth graders. 


Illustrative Item 


9 MINUTES 


Some scientists study 
space with large helium 


gas balloons. These 
balloons are usually 
launched from the ground 


into space but can also be 

launched from spacecraft 

near other planets 

Why do scientists use Enter your search below: 

these gas balloons to Tips for searching 

explore outer space and [gesbolioons SCSC~*~<“‘—s~—SSSSSSSSSSTTTCTCSCSOSSCS ; 

the atmosphere instead of es Semone | 

using satellites, rockets, or If you want to narrow your search, try placing the word 

other tools? Be sure to "and" between your search terms. 

explain at least three 

advantages of using gas 

balloons. 

Base your answer on more Documents 1 to 15 of 300 matching the query "gas balloons". 

than one web page or site 

Be sure to write your B 

answer in your own words! 

Abstract: Gas Balloons. From Our Exclusive Collection, Gas Balloons 97 - 1. Gas 
You will be scored on Balloons 97 - 2. Gas Balloons 97 - 3. Gas Balloons 97 - 4. Gas Balloons 97 - 5. 
Gas Balloons 95. Copyright © 1995-2000 by Westward Connections Inc. and/or 

« how well you search, respective content providers. Trade names and trademarks are recognized as 
the quality of your properties of the 


bookmarks, and 
« how well you answer 
the questions 


ef re rd wus 
size 0 3464 Fayien= 1/9/2003 ra 22: 33 PM GMT 


Copy = Ctrl+c _ ae a 
Paste = Ct+v 
Find = Ctrl +f 


(See appendix C for more information about this task.) 


P8.16, Using Scientific Inquiry 
Source: Persky et al. 2005. 
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Empirical investigation items put hands-on performance tasks on the computer and invite stu- 
dents to design and conduct a study to draw inferences and conclusions about a problem. Whether 
the computer-simulated experiment assesses the same skills, knowledge, and understanding as a 
hands-on performance task has not been established; a special study will be conducted to address 
this question (see p. 117). Following is a screen shot from a computer version of the electric myster- 


ies task (see p. 111 and appendix C for more information about this hands-on performance task). 


Illustrative Item 


Notes: Box B has a bulb in it. 


P4.11, Using Scientific Inquiry 
Source: Shavelson, Baxter, and Pine 1991, p. 357. 


Simulation items model systems (e.g., food webs), pose problems of prediction and explanation 
about changes in the system, and permit students to collect data and solve problems in the system. 


The following screen shot comes from such a simulation ICT. 
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Illustrative Item 


High school (grade 12) students are asked to identify locations appropriate for solar power gener- 
ation. (See appendix C for a description of the full task.) To complete the task, students must: 


e Evaluate GIS map visualizations. 
e Compare and contrast visualizations of different types of data. 


e Use analytical extension to perform computations with visualization data. 


Following is an example of one of several tasks that a student completes: 


Your task is to identify two states that will have high annual solar energy and will be able to gener- 
ate the maximum amount of electricity from their solar panels. Name two states that you predict 
will have a good annual electrical yield. In the rest of this performance assessment, you will gen- 
erate visualizations and calculate which states will generate the best annual electricity yield from 


solar panels. 


The following screen shot provides an example of a student response: 


This visualization shows that the states with the mast annual solar energy 
| are Arizona and Nevada, This data alone, however, does not completely 
justify the location of the panels, due to temperature issues, 


E12.10, Using Technological Design 
Source: Quellmalz et al. 2004. 
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Finally, concept mapping can be done by providing concept terms and asking students to build 
propositions on the computer by linking pairs of terms with arrows and words or phrases. Following 


is a screen shot of a completed concept map. 


Illustrative Item 


In this task, middle and high school students used a customized software program to create con- 
cept maps. Students received 18 environmental science terms and 7 link labels.* Students could 
drag and drop these concepts onto the grid space of the mapping program and add, erase, and link 
the items in their newly constructed maps. (See appendix C for more information about this task.) 


oO Concept Mapper 
File Add Concept Available Links 
producer 
decomposition 
oceans | 
fa evaporation 
requires t 
produces 

causes 

nutrients oxygen 

part of 
} requires food chain photosynthesis 
influences 
carbon dioxide | | 

sunlight 


L12.5, Identifying Science Principles 
Source: Adapted from Herl et al. 1999. 


* Link labels should not be provided to students on the NAEP Science Assessment. See the Specifi- 


cations for more information. 


Computers and other media provide potential solutions to a variety of practical challenges 
posed by complex assessment exercises. The logistical challenges of hands-on performance tasks can 
be circumvented with computer simulation. Extensive databases can be used to assess students’ abil- 
ity to select and evaluate information relevant to the situation or problem they need to address. In 
addition, the difficulty of providing materials and training for complex tasks (such as concept maps) 
can be circumvented with computers. To avoid cheating and teaching to the concept map, concept 


terms can be randomly sampled for a particular map (Ruiz-Primo et al. 2001). 
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ICTs should be used when the format offers advantages over other assessment modes. ICTs 
include (but are not limited to) testing student knowledge, skills, and abilities related to the follow- 
ing situations: 


e For scientific phenomena that cannot easily be observed in real time, such as seeing things 
in slow motion (e.g., the motion of a wave) or at a higher speed (e.g., erosion caused by a 
river). They are also useful when it is necessary to freeze action or replay. 

e For modeling scientific phenomena that are invisible to the naked eye (e.g., the movement 
of molecules in a gas). 

e For working safely in lab-like simulations that would otherwise be hazardous (e.g., using 
dangerous chemicals) or disorderly in an assessment situation. 

e For situations that require several repetitions of an experiment in limited assessment time 
while varying the parameters (e.g., rolling a ball down a slope while varying the mass, the 
angle of inclination, or the coefficient of friction of the surface). 

e For searching the Internet and resource documents that provide high-fidelity situations 
related to the actual world in which such performances are likely to be observed. 

e For manipulating objects in a simplistic manner, such as moving concept terms in a concept 


map. 


Extended constructed-response items, concept-mapping tasks, and simulated performance tasks 
are especially strong candidates for ICTs. In this way, the complex science understandings and prac- 
tices that need to be probed in the NAEP Science Assessment could be captured with less time, cost, 
and logistical challenges and with greater opportunity for divergent problem-solving tasks than has 


been the case in the past. 


Distribution of Items 


This section recommends appropriate item distributions. The three types of distribution, as 


measured by percentage of student response time at each grade level, are as follows: 


e Items by content area—Physical Science, Life Science, and Earth and Space Sciences 
e Items by science practice—Identifying Science Principles, Using Science Principles, Using 
Scientific Inquiry, and Using Technological Design 


e Items by type—Selected-response items and constructed-response items 


Distribution of Items by Content Area 


In the Overview of the Science Assessment section of this chapter, the distribution of items at 
each grade level by the three science content areas, as measured by percentage of student response 


time, was described as shown in exhibit 17. 
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Exhibit 17. Distribution of items by content area and grade 


Grade 4 Grade 8 Grade 12* 
(percentage of (percentage of (percentage of 
student response time) | student response time) | student response time) 


Physical Science 33.3 30.0 37.5 | 
Life Science 33.3 30.0 37.5 | 
Earth and Space Sciences 33 40.0 25.0 | 


* These distributions are based on NAEP data regarding students’ course-taking patterns in 12th grade. If 
these patterns change substantially, the distributions should be reconsidered. 


Distribution of Items by Science Practice 


The item distribution for the four science practices, as measured by percentage of student 


response time at each grade level, should have the following approximate allocations: 


Exhibit 18. Distribution of items by science practice and grade 


Grade 4 Grade 8 Grade 12 
(percentage of (percentage of (percentage of 
student response time) | student response time) | student response time) 


Identifying Science Principles 30 25 20 | 


30 35 40 
Using Scientific Inquiry 30 30 30 


Using Science Principles 


Using Technological Design 10 10 10 | 


Distribution of Items by Item Type 


As measured by student response time, 50 percent of the assessment items at each grade level 
should be selected-response items and 50 percent should be constructed-response items (short 
constructed response, extended constructed response, concept-mapping tasks). Individual selected- 
response items and individual constructed-response items used within each of the item clusters, 
POE item sets, hands-on performance tasks, and ICTs should be included in this distribution. If 
variation from this distribution becomes necessary as items are developed, preference should be 


given to constructed-response items. 


In regard to the combination item types, the NAEP Science Assessment should contain at least 


one of each of the following at each grade level: item clusters, POE item sets, hands-on performance 
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tasks, and ICTs. In addition, it is highly recommended that each assessment include at least one con- 


cept-mapping task in grades 8 and 12. 


Hands-on performance tasks and ICTs are administered to a subset of the students sampled; the 
quantity for each task is specified in exhibit 19. 


Exhibit 19. Distribution of items by type of item and grade 


Grade 4 Grade 8 Grade 12 
(number of tasks) (number of tasks) (number of tasks) 
Hands-On Performance Task >1 >1 >1 
Interactive Computer Task 21 21 21 
Total 24 24 >4 


In any grade, the number of ICTs should not exceed the number of hands-on performance tasks. 


No student will be administered both types of tasks. 


Students With Disabilities and English Language Learners 


Along with the increase in national and state testing, there is an increased demand for assess- 
ment systems to include all students (for example, those with disabilities and those learning English, 
many of whom have not been included in these systems in the past). As NAEP seeks to measure the 
educational progress of students in the nation’s classrooms, assessment developers will encounter 
challenges that require careful thought and consideration about the development of items that pro- 


vide the most equitable context possible for all students. 


NAEP should strive to develop science assessments that allow the widest possible range of 
students to participate. In this way, the interpretation of scores of all participants leads to valid 
inferences about the levels of their performance as well as valid comparisons across states and with 
state assessments. All students should have the opportunity to demonstrate their knowledge of the 


concepts and ideas that the NAEP Science Assessment is intended to measure. 
According to the National Research Council: 
Fairness, like validity, cannot be properly addressed as an afterthought once the test has 
been developed, administered, and used. It must be confronted throughout the intercon- 


nected phases of the testing process, from test design and development to administration, 


scoring, interpretation, and use (1999b, pp. 80-81). 
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When assessments are first conceptualized, they must be thought of in the context of the entire 
population that will be assessed (American Educational Research Association, American Psycholog- 
ical Association, and National Council on Measurement in Education 1999; NRC 1999a; Thomp- 
son, Johnstone, and Thurlow 2002). NAEP assessments, as well as all large-scale assessments, must 
be responsive to growing demands: increased diversity, increased inclusion of all types of students in 
the general curriculum, and increased emphasis and commitment to serve and be accountable for all 
students. Assessments must measure the performance of students who have a wide range of abilities 
and skills, ensuring that those with diverse learning needs receive opportunities to demonstrate com- 


petence on the same content as all other students. 


Students with disabilities and English language learners present challenges in how their science 
competencies can be assessed in a valid manner. Nevertheless, these groups share some commonali- 


ties, including considerable heterogeneity in each group regarding assessment needs. In addition: 


e Conceptual frameworks based on appropriate theories of language development and profi- 
ciency and various forms of disabilities will be needed to build inclusive assessments. 
e A greater commitment to financial and human resources will be needed to develop and 


administer tasks and to interpret performance on tasks. 


Two general recommendations pertain to both groups in the context of good assessment design 


for all students: readability of written text and alignment to content statements. 


Students ability to read and respond to written text often determines successful performance 
on assessments. Assessment items may pose an unfair disadvantage for some students if there is a 
heavy burden on reading skills when reading is not the goal of the assessment. Language that is both 
straightforward and concise and that uses everyday words to convey meaning is needed. The goal of 
ensuring that language has these characteristics is to improve the comprehensibility of written text 
and at the same time preserve the essence of its meaning. Using language that reduces the linguistic 
demands placed on students reduces the effect of reading skills and language proficiency on stu- 
dents’ science performance and assessment scores. More information on reading level is provided in 


the specifications. 


Items on the NAEP Science Assessment must be aligned to the content statements and science 
practices with the same depth and breadth of coverage and the same cognitive demands as speci- 
fied in the framework. Assessment design should emphasize accessibility using different formats, 
technologies, designs, and accommodations to include as many students as possible. It must be clear 
from the outset that, to be equitable, assessments need to measure the achievement of all students 


on the same content and achievement standards. 
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To these ends, field tests should sample every type of student expected to participate in the final 
assessment administration, including students with a wide range of disabilities, English language 
learners, and students across racial, ethnic, and socioeconomic lines. Field testing NAEP items with 
a broad range of students will not only help determine whether items are unclear, misleading, or 
inaccessible for certain groups of students, but will also help ensure that assessment procedures are 
accessible to students when the NAEP Science Assessment is fully implemented. More information 


about these recommendations can be found in the Specifications. 


NAEP strives to assess all students selected by its sampling process. Rigorous criteria are 
applied to minimize the number of English language learners and students with disabilities who are 
excluded from NAEP assessments. Participating students with special needs are permitted to use 


accommodations, as stated in current NAEP policy: 


All special-needs students may use the same accommodations in NAEP assessments that 
they use in their usual classroom testing unless the accommodation would make it impossi- 
ble to measure the ability, skill, or proficiency being assessed, or the accommodation is not 
possible for the NAEP program to administer (NCES 2005a). 


See the Specifications for more information on NAEP’s inclusion policy and permitted 


accommodations. 


Special Studies 


Special studies bearing on aspects of the NAEP Science Assessment are presented in this frame- 


work. Each study contributes to a further understanding of science assessment. 


Group | special studies have the highest priority: 


e “Exchangeability” of hands-on performance and interactive computer investigations 

e Impact of variation in item format and language demand on the performance of English 
language learners and students with disabilities 

e Computer adaptive testing to assess the development of student understanding of 


Earth systems 


Group 2 special studies have lower priority: 


e Knowing what students know about technological design 
e Extended investigations by students 


The order in which studies are listed does not imply priority within group 1 or group 2. Group 1 


special studies are described in the next section. Group 2 special studies are discussed in appendix D. 
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Group 1 Special Studies 
“Exchangeability” of Hands-on Performance and Interactive Computer Investigations 


Inquiry is at the heart of knowing and doing science. A fundamental aspect of inquiry is the 
design, conduct, and interpretation of empirical investigations to answer a question or test a hypoth- 
esis. Although a full assessment of inquiry is not possible on any test that is given on demand, 
hands-on performance investigations (HPIs) attempt to approximate this aspect of inquiry under 
time, space, cost, and logistical constraints. For this reason, HPIs have been a part of the NAEP 


Science Assessment since 1996. 


However, HPIs have been criticized as being expensive, logistically difficult, and too highly 
structured. On the other hand, ICTs or, in this case, interactive computer investigations (ICIs), 
are logistically simpler, less expensive, and less structured. Therefore, the purpose of this study is to 
explore whether ICIs and HPIs are exchangeable. The question is zo¢ whether ICIs could replace 
HPIs either on NAEP or in the classroom—they should not. Even if these two approaches produce 
similar performances and scores, each affords somewhat different opportunities; simulations cannot 
be exchanged with actual practice. This is a question of assessment: can the cost and logistical chal- 
lenges of HPIs be reduced with the use of ICIs and still measure the same competencies with as much 
reliability and validity? Some research suggests that the two methods of assessing student inquiry 
are exchangeable to a fair degree (e.g., Pine, Baxter, and Shavelson 1993; Rosenquist, Shavelson, and 
Ruiz-Primo 2000). However, further research is needed on several different investigations to provide 


a satisfactory answer for large-scale assessment. 


Specifically, this study would address the following research questions: 


e Does the choice of an ICI or HPI limit the questions that may be asked? Specifically, is 
there something of value in an HPI that cannot be asked ifan ICI is administered? 

e Are scores on HPIs and ICIs equally reliable? 

e Are scores on HPIs and ICIs of equal magnitude? 

e To what extent does performance on an HPI predict performance on an ICI in the same 
investigation? 

e Doscores on HPIs and ICIs correlate somewhat equally with scores on another measure of 
science inquiry or achievement? 

e Are similar thinking processes evoked by HPIs and ICIs? 

e Do the answers to these questions depend on individual differences among students (such 
as gender, English proficiency, race/ethnicity, socioeconomic status, and geographic loca- 
tion)? Variation in student access to computers is also an area of interest; this factor could 


be confused with other variables listed here. 


117 | 


Impact of Variation in Item Format and Language Demand on the 
Performance of English Language Learners and Students With Disabilities 


English language learners and students with disabilities do not tend to perform as well on 
standardized achievement assessments as other students. Recent studies, for example, have pointed 
to a systematic relationship between the linguistic complexity of the assessment and the test scores 
of English language learners (¢.g., Abedi 2003) and students with certain disabilities, such as those 
related to reading and information processing. Science assessments, with their heavy reliance on 
verbal skills, may exacerbate performance disparities. In cases where this relationship is demonstrable 
and test items use complex language, the differences become sources of measurement error and con- 
struct irrelevant variance; therefore, the nature of the assessment item must be addressed. Until this 
dimension of the assessment item is more clearly understood, any interpretation of the performance 
of English language learners or students with disabilities on a content assessment is problematic; for 


example, language proficiency and science understanding cannot be separated. 


Preliminary results from several studies of scaffolded science assessments that are designed to 
minimize language complexity and provide alternative response modalities—including graphic 
organizers or drawn representations of the concepts—indicate that research is needed to clarify the 
relationship between language complexity, scaffolded assessment items, and the performance of 


English language learners and students with disabilities. 


Specifically, this study would address the following research questions: 


e Can the language complexity of a content-based assessment be systematically measured? 

e Can content-based assessment items be designed to minimize the language demand while 
conserving the content information obtained? 

e Ifthe content-based assessment contains a graphic response modality, do English language 
learners and students with disabilities demonstrate higher understanding of the content 
concept being assessed relative to more linguistically demanding response modalities? 

e When the content-based assessment with a graphic response option is also computer based, 
is there a further benefit in terms of content concept conservation and the performance of 


these students? 


Computer Adaptive Testing To Assess the Development 
of Student Understanding of Earth Systems" 


A common critique of large-scale assessment is that its necessary reliance on easily scored, 


decontextualized, and simplified items has led to an impoverished range of potential learning 


‘' Currently, a number of states are using computerized testing, consisting largely of translating traditional paper-and-pencil items into computer-based 
delivery systems. The study suggested here, as well as the first study on p. 116, makes more extensive use of the capabilities inherent in computer-based 
assessments. 
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activities from which valid and reliable measures might be derived (Resnick and Resnick 1992). 
Attempts to find alternatives include the following: 


e The facets approach (Minstrell 1998), which posits a strong model of facets of student 
knowledge for certain science topic areas and uses coordinated sets of multiple-choice 
items to hone in on students’ particular conceptions and misconceptions; and 

e The progress variable approach (Masters, Adams, and Wilson 1990), which posits a 
learning progression and uses item response theory to scale students’ responses to (typically 
open-ended) items to estimate the part of the learning progression in which students will 


most likely be located. 


This special study combines the strengths of each of these approaches to develop a new type of 
“branching” item that can be used to investigate the more complex types of knowledge structures 
and complex procedural steps involving contingencies such as those common in inquiry-related 
contexts, and yet maintain the efficiency of traditional multiple-choice testing. Specifically, the fac- 
ets approach will contribute its strong knowledge structure and convenient scoring, and the progress 
variables approach will contribute the interpretational framework of the learning progression and 
the flexible statistical modeling available through recent advances in item response modeling (De 
Boeck and Wilson 2004). Together, these approaches allow the use of item bundles (such as those 
shown in exhibit 20) to provide both the usual result in terms of student ability estimation as well 
as potentially more educationally informative results such as the prevalence of particular classes of 


misconceptions among the student body. 
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Exhibit 20. Storyboard showing item design fora 
“branching” item bundle onions and atoms (adapted from Scalise 2004) 


Matter Composition: lons and Atoms Item Bundle 


1, Lead-based paint contains Pb2+ ions and lead pipes are made up of Pb atoms 
The main difference between Pb2+ ion and Pb atom is: 

A.They are basically the same. (go to question 2) 

B. They have a different number of electrons. (go to question 3) 

C.They have a different number of protons (3) 

D.They are different but not in the ways described. (go to question 2) 


E.| don't know (0) 
2, Choose the answer with which you most agree. 3. Pick the best answer below: 
Pb2+ ion and Pb atom are the same except: A.Pb2+ has 2 fewer electrons than Pb. 
A.Pb2+ has ionic bond and Pb has atomic bonds. (3) (go to question 4) 
B.Pb2+ ion and Pb are similar but used differently. (1) B.Pb2+ has a larger e- density cloud. (3) 
C.Pb2+ is a liquid, Pb is a sold. (2) C.Pb2+ is positively charged so has 
D.Pb2+ requires two Pb atoms. (2) 2 extra valence electrons. (4) 


4, Electrons can be thought of as arranged in shells or orbitals. 
Select the ground state electron configuration for carbon: 

A. 1s22s12p3 (7) 

B. 1522s22p2 (go to question 5) 

C.1s22s22p1 (6) 

D. None of these (5) 


5. Certain main group elements exhibit multiple oxidation states, including Group 14 
of which carbon is a member. It would seem that a possible oxidation state for carbon 
is +2, with two electrons available for bonding in half-filled 2px and 2py orbitals. Based 
on valence bond theory, show an electron configuration justifying the formation 

of four bonds for carbon rather than two and describe the type of orbitals formed. 
EXPLAIN your answer as fully as possible. 

(All respondents who reach this screen will be scored as an 8. 

Answers submitted here will be used to build distractors for this question.) 


Possible scores and parameters for this item bundle under iota model (assume constraint on cases): 
score 0 1 2 3 4 5 6 7 8 
# paths 1 1 2 3 1 1 


1 1 1 
parameters 67; 572 73,1731 = -574,1741.1742 575 576 577 57g 579 
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Specifically, this study would address the following research questions: 


e Can the “branching” item type be developed and delivered in a logistically efficient way for 
use in NAEP? 

e Can the information from sets of “branching” item bundles be used to provide reliable, 
valid, and useful information on both overall student ability in science and the classifica- 


tion of students into educationally useful categories? 


The study would focus on a specific Earth system that is of practical and environmental signifi- 
cance, such as the biogeochemical carbon cycle. Understanding this system and related environmen- 
tal issues (e.g., global climate change) requires connected understandings in the Physical, Life, and 
Earth and Space Sciences, many of which are characterized in the framework. For example, students 
who understand global warming understand how photosynthesis, cellular respiration, and fossil fuel 
combustion affect carbon dioxide concentrations in the atmosphere. This connected understanding 


can be tracked as a learning progression. 
While these three specific studies were not conducted due to resource constraints, NAEP con- 


tinues to pursue a broad program of research in areas related to innovative assessment tasks, testing 


special needs students, and computer adaptive testing. 
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APPENDIX A 


STEERING COMMITTEE GUIDELINES 


The Framework Is Informed by the National Standards and Benchmarks 


The framework should reflect the nation’s best thinking in science instruction; therefore, it is 
guided by two national documents: National Science Education Standards (NRC 1996) and Bench- 
marks for Science Literacy (AAAS 1993). Both documents were subject to extensive internal and 


external reviews during their development. 


Informed by the National Standards and Benchmarks, the framework should emphasize knowl- 
edge and use of science concepts, appropriate linking of science facts to concepts, relationships 
among concepts, and major themes unifying the sciences. The framework should also incorporate 


investigative skills. 


The Framework Reflects the Nature and Practice of Science 


The National Standards and Benchmarks include standards addressing science as inquiry, nature 
of science, history of science, and the designed world. The framework should emphasize the impor- 
tance of these aspects of science education and should include the expectation that students will 
understand the nature and practice of science. Science is a self-correcting process; that is, a way of 
knowing in which theories are continually modified and refined based on new research findings. 


Students should demonstrate the ability to accomplish the following: 


e Make warranted inferences from evidence. 

e Use evidence to justify conclusions based on scientific investigations. 

e Demonstrate reasoning skills in the application of science content and in understanding 
the connections between science concepts. 

e Exercise skepticism when evaluating, using, and discarding data. 

e Understand and use models to describe and do science. 

e Apply content knowledge and skills to solve problems as they occur in the natural world. 


e Understand and apply knowledge of links and commonalities of science across fields. 
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The scientific disciplines are no longer practiced in isolation, and research that cuts across disci- 


pline boundaries is common. The framework should: 


e identify some of the science concepts and skills that cut across the assessed content areas; 
e address science in both the natural and designed world; and 
e clearly define and identify commonalities and differences between “science” and “technol- 


ogy” or “technological design.” 


The framework should also address social and historical contexts, which are keys to understand- 


ing how the scientific community has arrived at its current body of knowledge. 


The Framework Incorporates Key Attributes of Effective Assessment 


The framework should use assessment formats that are consistent with the objectives being 


assessed. It should be guided by the best available research on assessment item design and delivery. 


The framework should take into account student diversity as reflected in gender, geographic 
location, language proficiency, race/ethnicity, socioeconomic status, and disability condition. The 
assessment should be designed and written so it is accessible by the majority of students and to min- 
imize the need for special accommodations for both students with disabilities and English language 
learners. Students with special needs should be provided accommodations so they can participate in 


the assessment. 


The framework should reflect knowledge about the acquisition of key science concepts over 
time, based on research about how students learn. The existing research findings should make clear, 
when possible, the progression of science knowledge across the grade levels. Concepts should be 
represented in a manner that reflects how students progress through a discipline and across disci- 
plines. Assessment items should reflect students’ potential for applying concepts and more varied 


and complex situations over time. 


Critical content and skills should be articulated and assessed across grades 4, 8, and 12 (verti- 
cally) and across the fields of science (horizontally) by creating items that are deliberately layered 
to achieve these goals. An example of measuring similar constructs within and across subjects is the 
progression of increasingly sophisticated understanding about energy from elementary school to 
middle school to high school in the content areas of biology, chemistry, Earth science, and physics. 


The NAEP assessment should continue to use a variety of assessment formats, including 
well-constructed multiple-choice and open-ended items as well as performance tasks. In addition, 
multiple methods of assessment delivery should be considered, including the appropriate uses of 
digital-based technology. The framework should consider using digital delivery systems for the 
assessment, including Web-based or CD-ROM formats. Embedded simulations that can represent 
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scientific phenomena such as data, representations, and factors captured within laboratory experi- 


ments, along with the use of an adaptively designed series of assessment items, should also be con- 


sidered. Advances in machine scoring of text should provide the opportunity for increased use of 


open-response format questions. The assessment format and delivery system employed should be 


accessible to the widest range of students. 


Each achievement level—NAEP Basic, NAEP Proficient, and NAEP Advanced—should include 


a range of items assessing various levels of cognitive knowledge that is broad enough to ensure each 


is measured with the same degree of accuracy. Descriptions of NAEP Basic, NAEP Proficient, and 
NAEP Advanced must be as clear as possible. 


The Assessment Provides Data for Research 


NAEP assessment results are increasingly being used to review state student assessments and 


compare student achievement across states. The framework should address the important uses of 


assessment data both to conduct research to better understand science learning and to improve sci- 


ence achievement. Data from the assessment should be collected in such a way as to provide infor- 


mation that accomplishes the following: 


supplies details of the attributes (e.g., race/ethnicity, gender) of the students being assessed; 
provides results by student gender, race/ethnicity, and socioeconomic level; 

describes the academic preparation of the teachers whose students are being assessed; 
describes the nature of the educational system of the students being assessed; 

relates the instructional delivery and materials, professional development of the teachers, 
and the learning environment to the results from the assessment; and 


provides feedback to educators for improving science instruction and learning. 


The Specifications Document Is Closely Aligned With the Framework 


The connections among the framework, the specifications, and the assessment items should be 


transparent, have a consistent level of specificity, and be coherent. 


The specifications should be written with consistent detail across all fields, domains, and expec- 


tations of the framework: 


The specifications should have a consistent structure across all areas. 

The expected science knowledge that represents the target for assessment should be 
described in a clear and consistent format. The specifications should address content that 
reflects the standards and focuses on the significant information and knowledge that stu- 
dents should retain over time (e.g., principle ideas, fundamental understandings). 

The specifications should use verbs that describe the expected action to be taken in the 


assessment (e.g., identify, describe, evaluate, relate, analyze, demonstrate). 
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e Expectations across the content areas should match in level of specificity and scope. 
e The specifications should follow the idea of learning trajectories. To assess overarching 
concepts or themes, the assessment specifications should reflect a layered understanding of 


growth in knowledge of the concepts. 
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APPENDIX B 


NAEP SCIENCE ACHIEVEMENT LEVEL DESCRIPTIONS 


Congress authorized the Governing Board to develop appropriate student achievement levels 
on NAEP. The achievement level descriptions are statements of what students should know and 
be able to do on NAEP at grades 4, 8, and 12. To fulfill its statutory responsibility, the Governing 
Board developed a policy to guide the development of achievement levels for all NAEP subjects. It 
identified three levels of achievement to provide the public, educators, and policymakers with infor- 
mation on student performance on NAEP. These levels —NAEP Basic, NAEP Proficient, and NAEP 
Advanced—are used as a primary means of reporting NAEP results to describe “how good is good 
enough” at grades 4, 8, and 12. 


Exhibit B-1 displays the Governing Board’s generic policy definitions for NAEP Basic, NAEP 
Proficient, and NAEP Advanced achievement that pertain to all NAEP subjects and grades. 


Exhibit B—1. Generic achievement level policy definitions for NAEP 


Achievement 
Level Policy Definition 


NAEP Advanced | This level signifies superior performance. 


This level represents solid academic performance for each grade assessed. Students 
reaching this level have demonstrated competency over challenging subject matter, 


NAEP Proficient including subject-matter knowledge, application of such knowledge to real-world 
situations, and analytical skills appropriate to the subject matter. 
NAEP Basic This level denotes partial mastery of prerequisite knowledge and skills that are funda- 


mental for proficient work at each grade. 


During the framework development process, the project committees are asked to develop 
preliminary achievement level descriptions, based on the generic policy definitions, to guide item 
development. The descriptions are designed to provide examples of what students performing at 
the NAEP Basic, NAEP Proficient, and NAEP Advanced achievement levels should know and be 
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able to do in terms of the science content and practices identified in the framework; they are used 
to ensure that a broad range of items (covering NAEP Basic, NAEP Proficient, and NAEP Advanced 
levels) is developed at each grade level. The NAEP assessment development contractor and item 
writers are the intended audiences for the descriptions. Planning committee members used their 
expert judgment to draft the preliminary descriptions and were partially guided by the following 


considerations: 


e NAEP Basic descriptions could be composed of both moderately challenging content and 
practice components. 

e NAEP Proficient descriptions could be composed of a mixture of highly and moderately 
challenging content and practice components. 

e NAEP Advanced descriptions could be composed of highly challenging science content 


and highly challenging components of a given science practice. 


The preliminary descriptions include only illustrative statements drawn from the framework’s 
science content and practices. The statements are not intended to represent the entire set of objec- 
tives from the content and practice dimensions, nor do the preliminary achievement level descrip- 
tions denote a sense of priority or importance based on the statements selected. In addition, the 
descriptions should not be interpreted as assessment items or as suggestions for assessment items. 
Although adjectives are not explicitly included in every statement, each was written with the expec- 
tation that student performances would be “good” and “reasonable.” For example, the phrase “design 
an investigation” implies “design an organized and logical investigation” and “propose a framework” 


implies “propose a rational framework. 


The achievement level descriptions are a set of paragraphs derived from the preliminary achieve- 
ment level descriptions, and these paragraphs are used to report the NAEP science results to the 
general public and other audiences. For each grade level, the paragraphs describe what students 
should know and be able to do at the NAEP Basic, NAEP Proficient, and NAEP Advanced levels 
in terms of the science content and practices identified in the framework. After the assessment 
is administered, broadly representative panels engage in a standard-setting process to determine 
the achievement level cut scores on the NAEP scale. The cut scores represent the minimum score 


required for performance at each NAEP achievement level. 


NAEP Science Achievement Level Descriptions 


The achievement levels are cumulative; therefore, students performing at the NAEP Proficient 
level also display the competencies associated with the NAEP Basic level, and students at the NAEP 
Advanced \evel also demonstrate the skills and knowledge associated with both the NAEP Basic and 
the NAEP Proficient levels. The cut score indicating the lower end of the score range for each level is 
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noted in parentheses. Bold text is a short, general summary to describe performance at each achieve- 


ment level. 


NAEP Science Achievement Levels—Grade 4 


NAEP 
Basic 


Students performing at the NAEP Basic level should be able to describe, measure, 
and classify familiar objects in the world around them, as well as explain and make 
predictions about familiar processes. These processes include changes of states of 
matter, movements of objects, basic needs and life cycles of plants and animals, 
changes in shadows during the day, and changes in weather. They should be able to 
critique simple observational studies, communicating observations and basic mea- 
surements of familiar systems and processes, and look for patterns in their observa- 
tions. With regard to scientific constraints, they should also be able to propose and 


critique alternative solutions to problems involving familiar systems and processes. 


Science Practices: Students performing at the NAEP Basic level should be able to 
describe, measure, and classify familiar objects in the world around them, as well as 
explain and make predictions about familiar processes, using evidence to support their 
observations and conclusions. They should be able to critique simple observational 
studies, communicate observations and basic measurements of familiar systems and pro- 
cesses, and look for patterns in their observations. They should also be able to propose 


and recognize alternative solutions to problems involving familiar systems and processes. 


In the physical sciences, students performing at the NAEP Basic level should be able to 
describe the properties of the states of matter, describe how to change matter from one 
state to another, describe different forms of energy, predict the electrical energy transfers 
that will take place in a simple circuit, critique alternative explanations for changes in a 
moving object’s position, and design an investigation to show how exerting a force on an 


object changes the object’s motion. 


In the life sciences, students performing at the NAEP Basic level should be able to identify 
the stages in the life cycles of familiar organisms; describe how familiar animals meet 
their basic needs for food, air, water, and shelter; observe and describe the changes in 
plants and animals during their life cycles; and describe how environments meet the 


survival needs of familiar plants and animals. 


In the Earth and space sciences, students performing at the NAEP Basic level should be 
able to predict changes in the length and position of shadows cast by the sun, describe 
how slow Earth processes (e.g., erosion) and fast Earth processes (e.g., volcanic eruption) 
can change Earth’s surface, distinguish between natural and manmade materials, choose 
and use a tool to monitor how weather conditions change, and identify Earth resources 
that are limited. 
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NAEP 
Proficient 
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Students performing at the NAEP Proficient level should be able to demonstrate 
relationships among closely related science concepts, as well as analyze alternative 
explanations or predictions. They should be able to explain how changes in tem- 
perature cause changes of state, how forces can change motion, how adaptations 
help plants and animals meet their basic needs, how environmental changes can 
affect their growth and survival, how land formations can result from Earth pro- 
cesses, and how recycling can help conserve limited resources. They should be able 
to identify patterns in data and/or explain these patterns. They should also be able 


to identify and critique alternative responses to design problems. 


Science Practices: Students performing at the NAEP Proficient level should be able to 
demonstrate relationships among closely related science concepts and familiar phenom- 
ena around them, as well as analyze alternative explanations or predictions, using evi- 
dence to support their explanations and predictions; critique observational studies and 
simple investigations; identify patterns in data and/or explain those patterns in data; and 
apply scientific ideas to identify and critique alternative designs to problems that person- 


ally affect them. 
In the physical sciences, students performing at the NAEP Proficient level should be able 


to demonstrate the relationship between temperature change and changes in the physical 
properties of matter, explain how energy in one form can be changed into another form, 
design an investigation that measures how temperature changes when energy is added 

to a substance, propose a design for a container that will maintain the temperature of an 
object that is above or below room temperature, and measure changes in position of an 


object in motion as different forces are applied. 


In the life sciences, students performing at the NAEP Proficient level should be able 

to describe needs of familiar plants and animals at different stages of their life cycles, 
explain adaptations of familiar plants and animals to their environments, predict effects 
of environmental changes on plant or animal growth and survival, and apply informa- 


tion about an animal’s basic needs to propose a supportive environment. 


In the Earth and space sciences, students performing at the NAEP Proficient level should 
be able to explain how the Sun’s changing position in the sky during the day affects shad- 
ows; interpret land formations as resulting from either slow (e.g., erosion) or rapid (e.g., 
volcanic eruption) Earth processes; explain how natural materials can help sustain the 
lives of familiar plants and animals; identify how patterns of weather conditions change 
from season to season; and explain how the practices of recycling, reusing, and reducing 


help to conserve limited resources. 


NAEP 
Advanced 


Students performing at the NAEP Advanced level should be able to demonstrate 
relationships among different representations of science principles, as well as pro- 
pose alternative explanations or predictions of phenomena. They should be able 

to use numbers, drawings, and graphs to describe and explain motions of objects; 
analyze how environmental conditions affect growth and survival of plants and 
animals; describe changes in the Sun’s path through the sky at different times of 
year; and describe how human uses of Earth materials affect the environment. They 
should be able to design studies that use sampling strategies to obtain evidence. 
They should also be able to propose and critique alternative individual and local 


community responses to design problems. 


Science Practices: Students performing at the NAEP Advanced level should be able to 
demonstrate relationships among different representations of principles, as well as pro- 
pose alternative explanations or predictions of familiar phenomena, using evidence to 
support their explanations and predictions; design observational studies or simple inves- 
tigations to validate or criticize explanations or predictions and use sampling strategies 
to obtain evidence; and propose and critique alternative individual and local community 


responses to design problems. 


In the physical sciences, students at the NAEP Advanced level should be able to demon- 
strate the relationship between the quantity of energy needed to change the state of a 
sample of a substance and the weight of the sample, demonstrate how different represen- 
tations (i.e., verbal, numerical, graphical) can be used to show the motion of an object, 
suggest an example of how the motion of an object can be changed without touching it, 
and design an investigation that demonstrates how long it takes different forms of energy 


to change the temperature of matter. 


In the life sciences, students at the NAEP Advanced level should be able to evaluate rela- 
tionships between changing environmental conditions and organisms’ growth, survival, 
and reproduction; analyze environments for how they may have different effects on the 
growth and survival of plants or animals of the same kind; and investigate the relation- 


ship between light and plant growth. 


In the Earth and space sciences, students at the NAEP Advanced level should be able to 
relate changes in the Sun’s daily path through the sky to different times of year, suggest 
examples of Earth materials that can be modified to meet human needs, explain how 
erosion is caused by daily/seasonal weather events, propose methods of reducing the 
amount of erosion, describe how humans can change environments that can be either 
detrimental or beneficial for themselves and other organisms, and describe how the use 


of Earth materials by humans impacts the environment. 
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NAEP Science Achievement Levels—Grade 8 


NAEP 
Basic 
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Students performing at the NAEP Basic level should be able to state or recognize 
correct science principles. They should be able to explain and predict observations 
of natural phenomena at multiple scales, from microscopic to global. They should be 
able to describe properties and common physical and chemical changes in materials; 
describe changes in potential and kinetic energy of moving objects; describe levels 
of organization of living systems—cells, multicellular organisms, and ecosystems; 
identify related organisms based on hereditary traits; describe a model of the solar 
system; and describe the processes of the water cycle. They should be able to design 
observational and experimental investigations employing appropriate tools for mea- 
suring variables. They should be able to propose and critique the scientific validity of 


alternative individual and local community responses to design problems. 


Science Practices: Students performing at the NAEP Basic level should be able to state or 
recognize correct science principles; explain and predict observations of natural phe- 
nomena at multiple scales, from microscopic to global, using evidence to support their 
explanations and predictions; design investigations employing appropriate tools for mea- 
suring variables; and propose and critique the scientific validity of alternative individual 


and local community responses to design problems. 


In the physical sciences, students at the NAEP Basic level should be able to recognize a 
class of chemical compounds by its properties; design an investigation to show changes 
in properties of reactants and products in a chemical process such as burning or rusting; 
describe the changes in kinetic and potential energy of an object such as a swinging 
pendulum; describe and compare the motions of two objects moving at different speeds 
from a table of their position and time data; describe the direction of all forces acting on 
an object; and suggest an example of a system in which forces are acting on an object but 


the motion of the object does not change. 


In the life sciences, students at the NAEP Basic level should be able to identify levels of 
organization within cells, multicellular organisms, and ecosystems; describe how changes 
in an environment relate to an organism's survival; describe types of interdependence 

in ecosystems; identify related organisms based on hereditary traits; discuss the needs 

of animals and plants to support growth and metabolism; and analyze and display data 


showing simple patterns in population growth. 


In the Earth and space sciences, students at the NAEP Basic level should be able to 
describe a Sun-centered model of the solar system that illustrates how gravity keeps the 
objects in regular motion; describe how fossils and rock formations can be used as evi- 
dence to infer events in Earth’s history; relate major geologic events, such as earthquakes, 
volcanoes, and mountain building to the movement of lithospheric plates; use weather 
data to identify major weather events; and describe the processes of the water cycle 


including changes in the physical state of water. 


NAEP 
Proficient 


Students performing at the NAEP Proficient level should be able to demonstrate rela- 
tionships among closely related science principles. They should be able to identify 
evidence of chemical changes; explain and predict motions of objects using posi- 
tion-time graphs; explain metabolism, growth, and reproduction in cells, organisms, 
and ecosystems; use observations of the Sun, Earth, and Moon to explain visible 
motions in the sky; and predict surface and groundwater movements in different 
regions of the world. They should be able to explain and predict observations of phe- 
nomena at multiple scales, from microscopic to macroscopic and local to global, and 
to suggest examples of observations that illustrate a science principle. They should be 
able to use evidence from investigations in arguments that accept, revise, or reject sci- 
entific models. They should be able to use scientific criteria to propose and critique 
alternative individual and local community responses to design problems. 


Science Practices: Students performing at the NAEP Proficient level should be able to 
demonstrate relationships among closely related science principles; explain and predict 
observations of phenomena at multiple scales, from microscopic to macroscopic and 
local to global, and to suggest examples of observations that illustrate a science principle; 
design investigations requiring control of variables to test a simple model, employing 
appropriate sampling techniques and data quality review processes, and use the evidence 
to communicate an argument that accepts, revises, or rejects the model; and propose and 
critique solutions and predict the scientific validity of alternative individual and local 


community responses to design problems. 


In the physical sciences, students at the NAEP Proficient level should be able to 
demonstrate the relationship between the properties of chemical elements and their 
position on the periodic table; use empirical evidence to demonstrate that a chemical 
change has occurred; demonstrate the relationship of the motion of an object that 
experiences multiple forces with the representation of the motion on a position-time 
graph; predict the position of a moving object based on the position-time data presented 
in a table; and suggest examples of systems in which potential energy is converted into 
other forms of energy. 


In the life sciences, students at the NAEP Proficient level should be able to explain metab- 
olism, growth, and reproduction at multiple levels of living systems: cells, multicellular 
organisms, and ecosystems; predict the effects of heredity and environment on an organ- 
ism’s characteristics and survival; use sampling strategies to estimate population sizes in 
ecosystems; and suggest examples of sustainable systems for multiple organisms. 


In the Earth and space sciences, students at the NAEP Proficient level should be able to 
explain how gravity accounts for the visible patterns of motion of the Earth, Sun, and 
Moon; explain how fossils and rock formations are used for relative dating; use models of 
Earth’s interior to explain lithospheric plate movement; explain the formation of Earth 
materials using the properties of rocks and soils; identify recurring patterns of weather phe- 
nomena; and predict surface and groundwater movement in different regions of the world. 
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NAEP 
Advanced 
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Students performing at the NAEP Advanced level should be able to develop alter- 
native representations of science principles and explanations of observations. They 
should be able to use information from the periodic table to compare families of 
elements; explain changes of state in terms of energy flow; trace matter and energy 
through living systems at multiple scales; predict changes in populations through 
natural selection and reproduction; use lithospheric plate movement to explain geo- 
logical phenomena; and identify relationships among regional weather and atmo- 
spheric and ocean circulation patterns. They should be able to design and critique 
investigations involving sampling processes, data quality review processes, and con- 
trol of variables. They should be able to propose and critique alternative solutions 
that reflect science-based trade-offs for addressing local and regional problems. 


Science Practices: Students performing at the NAEP Advanced level should be able to 
demonstrate relationships among different representations of science principles. They 
should be able to explain and predict observations of phenomena at multiple scales, from 
microscopic to macroscopic and local to global, and develop alternative explanations of 
observations, using evidence to support their thinking. They should be able to design 
control of variable investigations employing appropriate sampling techniques and data 
quality review processes that strengthen the evidence used to argue for one alternate 
model over another. They should be able to propose and critique alternative solutions 


that reflect science-based trade-offs for addressing local and regional problems. 


In the physical sciences, students at the NAEP Advanced level should be able to interpret 
diagrams, graphs, and data to demonstrate the relationship between the particulate 
nature of matter and state changes (for instance, melting and freezing); demonstrate 
relationships between position on the periodic table and the characteristics of families 
of the chemical elements; explain changes of state in terms of energy flow in and out of 
a system; identify possible scientific trade-offs in making decisions on the design of an 
electrical energy power plant; suggest examples of systems in which objects are undergo- 
ing transitional, vibrational, and rotational motion; and suggest examples of systems in 


which forces are acting both through contact and at a distance. 


In the life sciences, students at the NAEP Advanced level should be able to explain move- 
ment and transformations of matter and energy in living systems at cellular, organismal, 
and ecosystem levels; predict changes in populations through natural selection and 
reproduction; and describe an ecosystem’s populations and propose an analysis for 


changes based on energy flow through the system. 


In the Earth and space sciences, students at the NAEP Advanced level should be able to 
explain the seasons, Moon phases, and lunar and solar eclipses; illustrate how fossils and 
rock formations can provide evidence of changes in environmental conditions over time; 
use lithospheric plate movement to explain geological phenomena; identify relationships 
among regional weather and atmospheric and ocean circulation patterns; and use the 
water cycle to propose and critique ways for obtaining drinkable water. 


NAEP Science Achievement Levels—Grade 12 


NAEP 
Basic 


Students performing at the NAEP Basic level should be able to describe, measure, 
classify, explain, and predict phenomena at multiple scales, from atomic/molec- 
ular to interstellar. These phenomena include the structure of atoms and mole- 
cules; transformations of matter and energy in physical, Earth, and living systems; 
motions of objects; the genetic role of DNA; changes in populations and ecosys- 
tems due to selection pressures; earthquakes and volcanoes; patterns in weather 
and climate; and biogeochemical cycles. They should be able to design and critique 
observational and experimental studies, and they should be able to propose and 


critique solutions to problems at local or regional scales. 


Science Practices: Students performing at the NAEP Basic level should be able to 
describe, measure, classify, explain, and predict phenomena at multiple scales, from 
atomic/molecular to cosmic; design and critique observational and experimental 
studies, controlling single variables, making basic decisions about sampling, analyzing 
reliability of data, and using scientific models to explain results; and propose, critique, 


and predict scientific outcomes of responses to problems at local or regional scales. 


In the physical sciences, students at the NAEP Basic level should be able to explain the 
differences in atomic structure across families in the periodic table and explain how the 
structures of molecules change in chemical reactions; distinguish linear velocity and 
acceleration as each is represented graphically and suggest ways in which forces can be 
measured; critique data that claim to show how gravitational potential energy changes 
with distance from the Earth’s surface; predict the situations in which a net force 
changes the motion of an object; and predict how the energy packets of electromag- 


netic waves change as the frequency of the waves change. 


In the life sciences, students at the NAEP Basic level should be able to identify changes in 
populations due to selection pressures and trace matter and energy through organisms 
and ecosystems; explain changes in ecosystem structure and function and identify ways 
in which humans can permanently alter ecosystems through intentional design or unin- 
tended consequences; and describe the relationship between DNA and an individual's 


hereditary traits. 


In the Earth and space sciences, students at the NAEP Basic level should be able to 
describe a Sun-centered model of the solar system that illustrates how gravity keeps 
objects in regular motion; describe how fossils and rock formations can be used as 
evidence to infer events in Earth’s history; relate major geologic events, such as earth- 
quakes, volcanoes, and mountain building to the movement of lithospheric plates; use 
weather data to identify major weather events; and describe the processes of the water 


cycle, including changes in the physical state of water. 
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Students performing at the NAEP Proficient level should be able to demonstrate 
relationships and compare alternative models, predictions, and explanations. They 
should be able to explain trends among elements in the periodic table; conservation 
laws; chemical mechanisms for metabolism, growth, and reproduction; changes in 
populations due to natural selection; the evolution of the Universe; and evidence for 
boundaries and movements of tectonic plates. They should be able to design and cri- 
tique observational and experimental studies, controlling multiple variables, using 
scientific models to explain results, and choosing among alternative conclusions 
based on arguments from evidence. They should be able to compare scientific costs 


or risks and benefits of alternative solutions to problems at local or regional scales. 


Science Practices: Students performing at the NAEP Proficient level should be able to 
describe, measure, classify, explain, and predict phenomena at multiple scales, from 
atomic/molecular to cosmic; demonstrate relationships and compare alternative mod- 
els, predictions, and explanations; design and critique observational and experimental 
studies, controlling multiple variables, making basic decisions about sampling, analyz- 
ing reliability of data, using scientific models to explain results, and choosing among 
alternative conclusions based on arguments from evidence; and compare scientific costs 


or risks and benefits of alternative solutions to problems at local or regional scales. 


In the physical sciences, students at the NAEP Proficient level should be able identify the 
unique properties of water and their implications for Earth’s organisms and climate; 
describe the pattern of data expected within a family of elements from the periodic 
table; predict the nature of an unbalanced force on an object by the object’s motion 
and describe observations that would imply a conservation principle in science; suggest 
examples of how energy gets transferred in different processes; and design an experi- 
ment that will yield the average speed of an object under a free-fall situation. 


In the life sciences, students at the NAEP Proficient level should be able to explain 
chemical mechanisms for metabolism, growth, and reproduction in living systems; 
analyze cases of evolutionary change in populations using the following related science 
principles: the potential of a species to increase its numbers, the genetic variability of 
its offspring, limitations on the resources required for life, and the ensuing selection of 
those organisms better able to survive and leave offspring; and use scientific models to 


explain data patterns related to metabolism, genetics, or changes in ecosystems. 


In the Earth and space sciences, students at the NAEP Proficient level should be able to 
describe the theory that the Universe expanded from a single point billions of years ago 
and that most elements are formed in stars; given data about fossils, reconstruct the pos- 
sible environment in which the organisms lived; select geologic data to infer Earth’s tec- 
tonic plate boundaries; explain the factors that affect regional climates; use knowledge 
of biogeochemical cycles to predict how an ecosystem may change due to pollutant or 
change in land use; and propose methods to lessen negative impacts on ecosystems. 


NAEP 
Advanced 


Students performing at the NAEP Advanced level should be able to use alternative 
models to generate predictions and explanations. They should be able to explain 
differences among physical, chemical, and nuclear changes; the wave and particle 
nature of light; paths of specific elements through living systems; responses of 
ecosystems to disturbances; evidence for the theory of an expanding Universe; and 
evidence for human effects on the Earth’s biogeochemical cycles. They should be 
able to design and critique investigations that relate data to alternative models of 
phenomena. They should be able to compare costs or risks and benefits of alterna- 


tive solutions to problems at local, regional, and global scales. 


Science Practices: Students performing at the NAEP Advanced level should be able to 
describe, measure, classify, explain, and predict phenomena at multiple scales, from 
atomic/molecular to cosmic; demonstrate relationships and use alternative models to 
generate predictions and explanations; design and critique observational and experimen- 
tal studies, controlling multiple variables, making complex decisions about sampling, 
analyzing reliability of data, using scientific models to explain results, and choosing 
among alternative conclusions based on arguments from evidence; and compare scien- 
tific costs or risks and benefits of alternative solutions to problems at local, regional, and 


global scales. 
In the physical sciences, students at the NAEP Advanced level should be able to describe 


how physical, chemical, and nuclear reactions differ; state the changes to a gas ina 
closed system with the addition of energy; suggest empirical evidence to demonstrate 
the conservation of matter in physical and chemical changes; describe energy trans- 
formations that occur in the transmission of electromagnetic waves and design an 
investigation to identify the characteristics of electromagnetic waves; demonstrate the 
relationship of mass and velocity in conserving momentum during a two-body colli- 
sion; analyze conflicting claims about scientific evidence related to issues such as effects 
of extended use of cell phones on the human brain and effective methods of contain- 
ment of nuclear waste materials; and critique an experimental setup that measures 


velocities of an object to obtain average acceleration. 


In the life sciences, students at the NAEP Advanced level should be able to predict 
changes in ecosystems in response to disturbances and trace elements through physical 
and chemical changes in cells, organisms, and ecosystems; analyze conflicting claims 
about scientific evidence related to biological issues such as genetically modified organ- 
isms and ecological effects of climate change; and design technological systems that 


mitigate harmful science-related effects on humans and ecosystems. 
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In the Earth and space sciences, students at the NAEP Advanced level should be able to 
cite evidence (e.g., red shift) that the Universe expanded from a single point billions 
of years ago and that all but the lightest elements are formed in stars; use data from an 
excavation site to infer the age of a fossil; explain the mechanisms for phenomena at 
plate boundaries by employing earthquake data and using conceptual models; identify 
scientific trade-offs among energy sources; analyze conflicting claims about scientific 
evidence related to water resource issues such as ground water contamination and 
effects of stream channelization, levees, or dams on flood control and flood plains; and 
apply knowledge of biogeochemical cycles to predict changes that may occur if there is 


a disturbance in Earth’s systems due to a pollutant or the removal of a natural resource 
in an ecosystem. 


APPENDIX C 


SAMPLE ITEMS AND SCORING GUIDES 


Item Source: TIMSS 1999, Grade 8 (framework, p. 69). 


Oak tree Caterpillar 


Look at the food web above. If the corn crop failed one year, what would most likely happen to the 


robin population? Explain your answer. 
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Note: A correct response must include a feasible explanation directly relating the predicted change 
in robin population to the effect of corn crop failure on prey/predator relationships indicated 
in the food web. Responses do not have to use the specific terms decrease, increase, and 
same, as long as the explanation is clear with respect to the effect on the robin population. If 
more than one effect is given, assign the code corresponding to the first correct explanation. 


| Correct Response 


-_ & 
i] 


Robin population may decrease. Explanation based on predators (snakes/hawks) eating 
more robins if mice die. 
Examples: Goes down. The mice would starve and the snake would eat the robins. 
There would be less robins because the mouse population would 
decrease and the snakes (and/or hawks) would eat more of the robins. 


Robin population may increase. Explanation based on predators (snakes/hawks) dying 
due to lack of food (mice). 
Examples: It would go up because the snakes die if the mouse starves. 
There could be more robins because there are fewer snakes (and/or 
hawks) to eat them. 


Robin population would stay the same with a feasible explanation. 
Example: — It would not change because the mouse would find other grain to eat 
so the snake would be unaffected. 


Other acceptable explanation. 


| Incorrect Response 


_ 
N 


Robin population would decrease. Incorrect explanation based on robins starving if 
snakes die (confuses prey/predator relationship). 
Examples: — Decreases because there are less snakes to eat. 
When corn dies, then snakes, then robins starve. 


=| 
—) 


Robin population would decrease. Incorrect explanation based on the robin needing 
corn to survive. 
Example: — Decrease because they need the corn. 


—l 
—_ 


Robin population would stay the same. Incorrect explanation based on the robins not 
needing corn to survive or not being connected to corn in the food web. (Does not 
consider the effect of predators.) 
Examples: — Nothing because the robin only eats insects. 
Nothing would happen. The corn is on a different chain in the food web. 


Mentions only that the whole food web will be upset and/or all the animals will die. 
Example: — The whole food web would erupt and everything would die. 


Other incorrect (including crossed out/erased, stray marks, illegible, or off task). 


| Nonresponse 


~l 


L739 
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Item Source: Colorado Department of Education 2002, Grade 8 (framework, p. 72). 


The graph below shows the distance traveled over time by a student walking down a hall. Use the 
information shown on the graph to do Numbers 7 and 8. 


Distance Traveled Over Time 


Distance (m) 


0 5 10 15 20 25 30 
Time (seconds) 


7. During which time interval was the student moving the fastest? 
OA 
OB 
Oc 
Cp 


Key: D 
8. What was the average speed of the student from 0 seconds to 5 seconds? 


Average speed: 


Scoring: Average speed is 2 m/s. (The student traveled 10 meters in 5 seconds.) 
of Se Sp 
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Item Source: Quellmalz et al. 2004 (framework, p. 75). 


Lynx/Hare Task 


This is an interactive computer task in which students are expected to conduct a scientific inves- 


tigation regarding the question of whether or not lynx should be introduced into a national park in 


order to reduce the abiding overpopulation of hares. Students are directed to complete six modules 


that use different computer programs to determine the best solution for the proposed question: 


Module 1 asks the student to access, organize, analyze, and interpret data they are given 
about the populations of hares over the past 4 years. For this module, students can use word 
processor, spreadsheet, or presentation software. 

Module 2 asks the student to determine a better way to analyze and display some disor- 
ganized data that show how many lynx and hares were present each year over the past 25 
years. 

Module 3 first asks the student to submit a Web search that will give insight into the rela- 
tionship between the lynx and hare populations. It subsequently asks the student to criti- 
cally evaluate the relevance of several supplied Web searches. 

Module 4 asks the student to collect information on specific questions regarding the lynx/ 
hare question, to take notes on the information given on the websites, and to include cita- 
tions for each site. 

Module 5 asks the student to use a modeling program to predict the results of adding more 
lynx to the parks through viewing population trends over several years. Students are then 
asked questions based on what they have observed in the modeling tool regarding increases 
or decreases in the hare population if lynx are added to the park or not added to the park. 
Module 6 asks the student to create a presentation with word processor or presentation 
software to communicate the problem, the findings, and any recommendations that 


resulted from the newly completed research. 


For this task, students receive scores for inquiry skills and technology use, and for the appropri- 


ate use of concepts within their explanations and recommendations. 


For more information, see http://ipat.sri.com/tasks/pred_prey/subtasks/taskstud.html. 
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Item Source: Washington Assessment of Student Learning 2004, Grade 8 (framework, p. 79). 


Occasionally, a fire will destroy a forest, burning down trees and pushing wildlife out of their forest 
homes. However, the forest will grow back. Eventually, through the process of forest succession as 


shown below, short grasses and flowers begin to grow and animals make new homes. 


Forest Succession 


Tall Shrubs 


Short grasses 
grasses Time 


Over time, shrubs and trees begin to grow. The forest returns to a lush habitat for the wildlife listed 
in the chart below. 


Forest Wildlife 
Ground dwelling Worms, beetles 
Reptiles and amphibians American toads, wood frogs, snakes, Eastern box turtles 
Small animals Squirrels, chipmunks 
Medium to large animals Opossums, raccoons, white-tailed deer, black bears 
Airborne Butterflies, moths, bees, wild turkeys, red-tailed hawks, bald 
eagles 


A power company owns part of a forest that was destroyed by a fire. The forest could take decades to 
rebuild on its own. The company’s department of environmental studies suggests planting new trees 


to help the forest rebuild. 
Using the information in the scenario: 


e Explain how planting trees could benefit the natural ecosystem. 


e Explain how planting trees could harm the natural ecosystem. 
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Scoring Rubric 


2-point response: The response demonstrates that the student can analyze how human societies’ 


use of natural resources affects the quality of life and the health of ecosystems. 


The student explains one reasonable way in which planting the trees could benefit the natu- 
ral ecosystem. 

AND 

The student explains one reasonable way in which planting the trees could harm the natural 
ecosystem. 


Example responses: 


Benefits of planting the trees include but are not limited to: 
e providing a habitat for animals; 
e providing a canopy, which would help prevent soil erosion; 
e creating root systems, which would anchor soil in place; and 
e creating shade, which would help maintain sunlight levels and inhibit the introduction of 


nonnative plant species. 


Harms of planting the trees include but are not limited to: 
e disrupting the natural flow of animals reentering the forest; 
e inhibiting the growth of other plants; 
e decreasing the diversity of tree species growing in the forest; and 
e introducing foreign species into an area, which may affect native species of plants and 


animals. 


1-point response: The response demonstrates that the student can partially analyze how human 


societies’ use of natural resources affects the quality of life and health of ecosystems. 


The student explains one reasonable way in which planting trees could benefit the 
environment. 

OR 

The student explains one reasonable way in which planting trees could harm the 


environment. 


0-point response: The response demonstrates that the student can perform little or no analysis of 


how human societies’ use of natural resources affects the quality of life and the health of ecosystems. 


Note: Benefits/harms to the natural ecosystem that only relate to humans will not be cred- 


ited score points. 


1144 


Annotated example of a 2-point response: 


Explanation of how planting trees could benefit the natural ecosystem: 
The ecosystem will rebuild more quickly, and that would give the 
animals a new habitat more quickly. 

Explanation of how planting trees could harm the natural ecosystem: 
It could disrupt the natural order of the ecosystem. If some 
smaller plants don’t get a chance to grow first, the trees might 


push them out. 


Annotation: 


The response demonstrates that the student can analyze how human societies’ use of natural 
resources affects the quality of life and health of ecosystems. 


The student explains one reasonable way in which planting the trees could benefit the natural 
ecosystem, “The ecosystem will rebuild more quickly, and that would give the animals a new habitat 
more quickly.” (1 point) The student explains one reasonable way in which planting the trees could 
harm the natural ecosystem, “If some smaller plants don’t get a chance to grow first, the trees might 


push them out.’ (1 point) 


Annotated example ofa 1-point response: 


Explanation of how planting trees could benefit the natural ecosystem: 


More plants and trees would mean giving off more oxygen, more shade, and 


more food for forest animals. 


Explanation of how planting trees could harm the natural ecosystem: 


There could be a very big storm and trees could fall on houses. 


Annotation: 


The response demonstrates that the student can partially analyze how human societies’ use of natural 


resources affects the quality of life and the health of ecosystems. 


The student explains one reasonable way in which planting the trees could benefit the natural eco- 
system: “More food for forest animals.’ (1 point) The student explains one way in which planting 
the trees could harm the human structure: “Trees falling on houses” but not the natural ecosystem. 


(0 points) 
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Annotated example of a 0-point response: 


Explanation of how planting trees could benefit the natural ecosystem: 
have more trees 


Explanation of how planting trees could harm the natural ecosystem: 


half the plants will die 


Annotation: 


The response demonstrates that the student can perform little to no analysis of how human societies’ 


use of natural resources affects the quality of life and the health of ecosystems. 


The student states one factor, “have more trees,” but does not explain one reasonable way in which 


planting the trees could benefit the natural ecosystem. (0 points) 


The student states a possible harm, “half the plants will die,” but does not explain how planting the 
trees could cause this harm to the natural ecosystem. (0 points) 


Item Source: NAEP 1996, Grade 8 (framework, p. 87). 


A space station is to be located between Earth and the Moon at the place where Earth’s gravitational 
pull is equal to the Moon’s gravitational pull. On the diagram below, circle the letter indicating the 
approximate location of the space station. 


Earth 


Moon 


Explain your answer. 


Scoring Rationale: Student demonstrates the ability to explain the role of gravity in a manmade 


satellite and relates the force of gravity to the mass (size) of the object pulling it. 


3 = Complete-Student circles point C and gives a correct explanation that gravitational pull 
depends on mass and distance; thus, the station must be closer to the Moon because the Moon’s 
mass is less than that of Earth. 


2 = Partial-Student circles point C and explains that the Moon has less gravity than Earth but does 
not link it to mass. 


1 = Unsatisfactory/Incorrect-Student circles A, B, or C and gives an incorrect explanation or no 


explanation. 
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Sample Student Responses 


Complete (Level 3) 


Explain your answer. 


Partial (Level 2) 


Explain your answer. 


Unsatisfactory/Incorrect (Level 1) 


Explain your answer. 


gravitational focus 
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Item Source: TIMSS 2003, Grade 4 (framework, p. 101). 


Ona hot, humid day the air contains a lot of water vapor. What happens to the water vapor in the 


air when the air becomes very cold? 


Note: Priority should be given to code 10. Ifa response mentions condensation or freezing, code 10 


should be the response even if other correct codes apply. Responses that mention ONLY that the 


water vapor becomes cold or rises without any mention of a change of state (explicitly or implicitly) 


are scored as incorrect (code 70 or 71). 


Code 


Response Item: $031382 


Correct Response 


10 


Refers to condensation or freezing (or equivalent). 
Examples: It freezes. 

It condenses. 

Condensation. 

It condenses and turns into rain. 


11 


Mentions cloud formation or a form of precipitation (e.g., rain, snow, fog, etc.). 
Examples: The water vapor changes to rain. 

It changes to snow. 

Water vapor turns into clouds. 

It rises into the clouds and becomes rain droplets. 


It turns foggy. 
It rains. 


19 


Other correct 
Examples: It falls to the ground. 


Incorrect Response 


70 


Mentions only that the water becomes cold. [No mention of a change of state or 
precipitation. ] 
Examples: The water vapor becomes cold. 

Its temperature drops. 


71 


Mentions only that water vapor rises (or similar). [No mention of condensation or 
precipitation. ] 
Example: The water vapor will rise on a hot day. 


7 


Other incorrect (including crossed out/erased, stray marks, illegible, or off task). 
Example: It disappears. 


Non-response 


99 


Blank 
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Item Source: New Standards Spring Field Test 1999 for High School (framework, p. 102). 


The biosphere (living organisms), lithosphere (rocks and soils of Earth’s crust), and atmosphere are 


all involved in the cycling of carbon atoms. Describe the role each plays in the carbon cycle. 


Scoring guide: 


Score Point | Description 


The response describes the cycling of carbon among all three reservoirs. The 
-: response is complete and detailed, showing evidence of logical reasoning. There is 


no evidence of misconceptions. 


The response describes the cycling of carbon between two of the reservoirs. The 


3 
response may contain omissions or minor errors. 
The response describes the presence of carbon compounds in two or more res- 

2 ervoirs but may not link the reservoirs. The response may contain errors or 
misconceptions. 

; The response is largely incomplete, lacks detail, and contains errors of fact and 
reasoning. 

‘ The response may contain words from the question but does not add any informa- 


tion that might answer the question. 


Off topic Off-topic response. 


Blank No marks were made in the student response section. 


Background information for appropriate response: 


Carbon forms a variety of compounds. Carbon monoxide (CO) and carbon dioxide (CO,) are 
carbon compounds that exist as gases in the atmosphere. Organic carbon compounds come in a 
wide variety of forms in living organisms: sugars, carbohydrates, cellulose, starches, collagen, chitin, 
amino acids, proteins, lipids, and nucleic acids. These organic compounds are also to be found in soil 
due to decaying and decomposition. The carbonate ion (CO,*) combines with different positive 


ions to form carbonate compounds found in rocks and soil. 


Producers (green plants) take carbon dioxide from the air and convert it to sugar, using solar energy 
(photosynthesis). Additional nutrients (including carbonate ions from the soil) are taken into the 
plant via its root system. These nutrients are rearranged and turned into the wide array of carbon 
compounds listed above, along with the sugar compounds formed via photosynthesis. As consum- 
ers eat producers, the carbon compounds are broken down and rearranged into carbon compounds 


used by the consumer organism. 
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When any living organism dies, the carbon compounds are broken down and returned to the soil by 
decomposer organisms. Huge sediment layers of dead marine organisms, which once had chitin cell 
walls or shells, compress under the weight of continuing sediments and the ocean. Uplifting of these 
layers forms sedimentary cliffs and rocks. Weathering and erosion continue to change the carbon 


compounds into forms that will again be incorporated into living organisms. 


Producer, consumer, and decomposer organisms harness the chemical bond energy stored in food 


via cellular respiration. Cellular respiration releases carbon dioxide back into the atmosphere. 


The score is based on evidence of: geochemical cycling of carbon. 
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Item Source: Shavelson, Baxter, and Pine 1991 (framework, pp. 107, 109). 


Electric Mysteries 


The following is a brief description of two warm-up tasks: 


1. Students are asked to connect one battery, one bulb, and wires so the bulb lights. They are then 
asked to draw a picture of this simple circuit. 


2. Given mystery box “?” students are asked to identify whether it contains a battery or a wire. 


They are told they can determine the contents of the mystery box by connecting it in a circuit 


with a bulb. 


The following is an excerpt from the main task instructions given to students: 


Find out what is in the six mystery boxes A, B, C, D, E, and F. They have five different things inside, 
shown below. Two of the boxes will have the same thing. All of the others will have something differ- 


ent inside. 


[ The five options—two batteries, a wire, a bulb, a battery and a bulb, nothing at all—are presented 


in words and drawings. Drawings are not provided here.] 


For each box, connect it in a circuit to help you figure out what is inside. You can use your bulbs, 


batteries, and wires in any way you like. 
When you find out what is in a box, fill in the spaces on the following pages. 


The following is an example of the student response format: 


Box A: Has inside. 


Draw a picture of the circuit that told you what was inside Box A: 


The following is a brief description of the scoring system: 


For each of the six boxes (A-F), students’ responses are scored on two components: (1) identifi- 
cation of the contents of the box, and (2) the circuit used to make the conclusion. For each box, if 
both components are correct, the student receives 1 point; if one or both components are incorrect, 


the student receives 0 points. Total maximum score is 6 points. 
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Item Source: Persky et al. 2005 (framework, p. 108). 


TRE Web Search Task 


In this task, students are asked to use a search engine to determine why scientists use large 
helium gas balloons to explore outer space and the atmosphere instead of using satellites, rockets, or 
other such tools. One open-ended question and several multiple-choice questions are presented to 
students. Students are scored on how well they performed the search, the quality of the bookmarked 


pages, and how well the questions were answered. 


A complex scoring framework is used to assess students’ proficiency on the task. A “student 
model” is created that describes the “theory” of how different skills are linked together. For this task, 
there are five component skills: problem solving in technology-rich environments, computer skill, 
scientific inquiry skill, exploration, and synthesis. An “evidence model” is created that describes how 
the students’ responses are connected to each of these skills. Once students begin the task, every 
action is recorded and connected to one or more skills in the student model. Then, a three-step 
process is used to evaluate this record. The first step is feature extraction, which shows what action 
the student took, when the action was taken, and any value that was associated with that action. The 
second step is feature evaluation, which gives the scores for the actions taken based on developed 
rules that show the best way to complete the task. The third step is evidence accumulation, which 
systematically combines responses into summary scores that detail the inferences that can be made 


from the students’ responses. 
This task incorporates a multifaceted evaluation process that can uncover the many skills 


involved in a single task or module. Thus, this module can provide a more comprehensive evaluation 


of students’ skills and aptitudes than traditional test questions. 
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Item Source: Quellmalz et al. 2004 (framework, p. 110). 


Solar Power Task 


This is an interactive computer task in which students act as energy consultants to identify 
which two states will generate the most amount of electricity from photovoltaic cells. There are four 
modules in which students use a series of map visualizations and other data to reach and to present 
their final conclusions. Ultimately, students are asked to create a presentation to one state, recom- 


mending that the state apply for federal funds marked for solar energy use. 


Module 1 asks the student to review and apply background information on the conditions that 
both optimize and reduce solar energy production. They are also asked to conduct simple analy- 


sis using the ArcView map visualization program. 


Module 2 asks the student to explore several datasets to identify states with high incoming solar 
radiation and to manipulate the ArcView program and its map calculator tool. The student uses 
the ArcView program to generate visualizations and to calculate which states will generate the 


best monthly and annual electricity yields from solar panels. 


Module 3 asks the student to determine what other data may be necessary to create the most 


compelling recommendation to the states. 

Module 4 asks the student to create a presentation to one of the two states determined to have 

the highest capacity for solar energy production. The student uses the newly gathered data to 

support the recommendation to the state. 

Both generic and item-specific rubrics are used for scoring; they include scores for content 
(math and science), use of problem-solving or inquiry strategies (planning and thorough communi- 


cation), and use of technological tools. 


For more information, see http://ipat.sri.com/tasks/solarpower/subtasks/solar_tasks.html. 
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Item Source: Adapted from Herl et al. 1999 (framework, p. 111). 


Concept-Mapping Task 


In this task, students use a custom software program to create a concept, or knowledge, map. 
Students are given 18 environmental science terms (atmosphere, bacteria, carbon dioxide, climate, 
consumer, decomposition, evaporation, food chain, greenhouse gases, nutrients, oceans, oxygen, 
photosynthesis, producer, respiration, sunlight, waste, and water cycle) and 7 link labels (causes, 
influences, part of, produces, requires, used for, and uses). Students can then drag and drop these 
concepts onto the grid space of the mapping program and add, erase, and link the items in their 
newly constructed maps. The concept maps are scored based on semantic content, organizational 


structure, number of terms used, and number of links made. 


Additionally, students explore a simulated World Wide Web space, which allows them to search 
for relevant environmental science information to improve their concept maps. Students can book- 
mark Web pages they believe are helpful in constructing their concept maps. This portion of the task 
is scored based on relevant information found, the hypertext links that were selected (browsing), 
keyword searching, and the accessing of three or more highly relevant Web pages for a single con- 


cept (focused browsing). 


While performing the concept-mapping task, students are able to access real-time feedback, 
which compares students’ maps to experts’ maps and gives corresponding feedback about which 
items are correct and which need improvement. A score is also assigned to how often a student 


accessed feedback, a measure of monitoring one’s learning. 
This computer-based assessment provides a detailed view into both the student’s ultimate perfor- 


mance and the steps of the thought processes that were employed to generate the ultimate product 


or answer (in this case, the concept map). 
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APPENDIX D 


GROUP 2 SMALL-SCALE SPECIAL STUDIES 


Knowing What Students Know About Technological Design 


Knowledge about technology and the technological design process are prominent in both the 
National Standards and Benchmarks. The National Standards state, “Although these are science edu- 
cation standards, the relationship between science and technology is so close that any presentation 
of science without developing an understanding of technology would portray an inaccurate picture 
of science” (NRC 1996, p. 190). Despite taking some time for schools to include technology in 
the curriculum for all students, there is growing recognition that technology should be an import- 
ant component. An increasing number of states include technology in their standards—30 states 
in 2001 and 38 states by 2004 (Meade and Dugger 2004). Consequently, NAEP is including the 
process of technological design as parallel to, but with less emphasis than, the process of scientific 


inquiry. 


Because relatively few questions on NAEP will probe the practice of Using Technological 
Design, this study proposes the development of an additional set of questions to probe students’ 


understanding of this practice in-depth. 
Specifically, this study would address the following research question: 
What do students know about technological design in the contexts of agricultural 
technologies, energy generation technologies, and technologies related to Earth materials 


and resources? 


This study was not conducted because the Governing Board added a new assessment of Tech- 


nology and Engineering Literacy in 2014 that covers this content. 


Extended Investigations by Students 


National and local science education standards emphasize scientific inquiry. In many states, this 


goal requires student engagement in projects that can take days, weeks, and even months as they 
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undertake genuine investigations. Important outcomes of these projects include a range of skills 
that are a crucial feature of high-quality science education but that cannot be assessed adequately in 
a 60-minute assessment (NRC 2005). They include, for example, gauging the quality of students’ 

(a) reasoning while framing their research questions; (b) planning for data collection and the execu- 
tion of that plan; (c) ability to meet unpredictable challenges that arise during any actual, ongoing 
scientific investigation; (d) persistence in seeking productive explanations for their observations and 
revising plans for the investigation; (e) lines of argument in deciding how to alter their experimental 
approach in the light of new evidence; (f) engagement with fellow students and/or the teacher in 
interpreting an observation or result and deciding what to do about it; and (g) deliberations when 


settling on the defensible conclusions that might be drawn from their work. 


In many countries, teachers are expected to assess students’ work during extended projects. 
Often their judgments of student achievement are made during ongoing classroom activities that are 
part of the regular curriculum. The teachers’ assessments are incorporated into an overall score that 
also includes results of the short, timed tests. In some countries, a defined percentage of the total 


score is based on teachers’ judgments about achievement associated with investigative projects. 


This study, then, might include both a national sample of students and an exploration of what 
other countries do under similar circumstances. Specifically, the study would address the following 
research questions: 


e What methods can be or have been developed to assess student achievement with respect 
to the ability to conduct extended scientific investigations? 

e To what extent are shorter investigations interchangeable for the extended investigation 
and to what extent are they not interchangeable? 


This study was not conducted because of lack of available funding. 
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